--- name: "uri" section: "3" platform: "linux" package: "liburi-dev" version: "2.13.0-3" release: "Debian 3.0 (Woody)" platforms: ["linux"] url: "https://new.carta.tech/uri" --- # uri(3) a set of functions to manipulate URIs ## NAME uri - a set of functions to manipulate URIs ## DESCRIPTION The header file for the library is **#include \** and the library may be linked using **-luri.** **uri** is a library that analyses URIs and transform them. It is designed to be fast and occupy as few memory as possible. The basic usage of this library is to transform an URI into a structure with one field for each component of the URI and vice versa. ## LIBRARY MODE The library behaviour is controled by the flags described bellow. The default set of flag is **URI\_MODE\_CANNONICAL\|URI\_MODE\_ERROR\_STDERR.** - **URI\_MODE\_CANNONICAL** All objects store URI in cannonical form. - **URI\_MODE\_LOWER\_SCHEME** The scheme of the URI is always converted to lower case. - **URI\_MODE\_ERROR\_STDERR** If an error occurs, the error string is printed on the STDERR chanel. - **URI\_MODE\_FIELD\_MALLOC** Each field may have its own malloc'd space. When the caller set a field it can assume the content of the field is saved in the object. Otherwise when the caller sets a field it must make sure that the memory containing the value of the field will not be freed before the object is deallocated. - **URI\_MODE\_FURI\_MD5** Use MD5 key calculated from the URL as a path name instead of the readable path name described in FURI chapter below. For example http://www.foo.com/ is transformed into the MD5 key 33024cec6160eafbd2717e394b5bc201 and the corresponding FURI is 33/02/4c/ec6160eafbd2717e394b5bc201. - **URI\_MODE\_URI\_STRICT** Behave in strict mode (see STRICTNESS below). - **URI\_MODE\_URI\_STRICT\_SCHEME** Behave in strict mode (see STRICTNESS below). - **URI\_MODE\_FLAG\_DEFAULT** The default mode of the library. ## STRUCTURE AND ALLOCATION The **uri\_t** type is a structure describing the URI. Access functions are provided and should be used to get the values of the fields and set new values. All the fields are character strings whose size is exactly the size of the string they contain. One can safely override the values contained in the fields, as long as the replacement string has a size lower or equal to the original size. If the replacement string is larger, the caller must use a buffer of its own. If the flag **URI\_MODE\_FIELD\_MALLOC** is not set, which is the default, the allocation policy for an **uri\_t** object is minimal. When an object is allocated using **uri\_alloc,** memory is allocated by the library to store the object. This memory will be released when the object is freed using **uri\_free.** When a field is set, the pointer is stored in the object and no copy of the string is kept. It is the responsibility of the caller to make sure that the string will live as long as the object lives. This policy is designed to prevent allocation as much as possible. Let's say you have a program that will operate on 50 000 URLs, only one malloc and a few realloc will be necessary instead of 50 000 malloc/free multiplied by the number of fields of the structure. The loop will look like this: ``` /* * Alloc an empty object. */ uri_t* uri = uri_alloc_1(); for(i = 0; i < 50000; i++) { /* * Reuse the object for another url, object grow * only if needed because the url is larger than * any previously seen url. */ uri_realloc(uri, url[i], strlen(url[i])); ... do something on uri ... /* * Print the url on stdout */ printf("%s\n", uri_uri(uri)); } ``` If the flag **URI\_MODE\_FIELD\_MALLOC** is set, each field will have a separatly allocated space, if necessary. The caller may assume that the object is always self contained and does not depend on externally allocated string. Each set function (uri\_scheme\_set, uri\_host\_set etc.) allocated the necessary space and duplicate the string given in argument. The **info** field contains flags that record which fields contain a malloc'd space and which does not (URI\_INFO\_M\_\* flags). This information is only valid between two calls of the library functions. For instance **uri\_cannonicalize** will reorganize allocated space. This policy is used for integration of the library into scripting langages such as Perl. - **info** A bit field carrying information about the URI. Each bit has a corresponding define that have the following meaning. **URI\_INFO\_CANNONICAL** Set if the URI is in cannonical form. **URI\_INFO\_RELATIVE** Set if the URI is a relative URI (does not start with {http,..}://). **URI\_INFO\_RELATIVE\_PATH** Set if the URI is a relative URI and the path does not start with a /. **URI\_INFO\_PARSED** Set if the URI was successfully parsed. If this flag is not set the content of the object is undefined. **URI\_INFO\_ROBOTS** Set if the URI is an http robots.txt file. **URI\_INFO\_M\_\*** There is such a flag for each field of the **uri\_t** structure. If the flag is set, the memory pointed by this field has been allocated by **malloc.** - **scheme** The scheme of the URI (http, ftp, file or news). - **host** The host name part of the URI. - **port** The port number associated to host, if any. - **path** The path name of the URI. - **params** The parameters of the URI (i.e. what is found after the ; in the path). - **query** The query part of a cgi-bin call (i.e. what is found after the ? in the path). - **frag** The fragement of the document (i.e. what is found after the # in the path). - **user** If authentication information is set, the user name. - **passwd** If authentication information is set, the password. ## FUNCTIONS - **uri\_t\* uri\_alloc\_1()** Allocate an empty object that must be filled with the **uri\_realloc** function. - **uri\_t\* uri\_alloc(char\* uri, int uri\_length)** The **uri** is splitted into fields and the corresponding **uri\_t** structure is returned. The structure is allocated using **malloc.** The URI is put in cannonical form. If it cannot be put in cannonical form an error message is printed on stderr and a null pointer is returned. - **uri\_t\* uri\_object(char\* uri, int uri\_length)** The **uri** is splitted into fields and the corresponding **uri\_t** structure is returned. The returned structure is statically allocated and must not be freed. The URI is put in cannonical form. If it cannot be put in cannonical form an error message is printed on stderr and a null pointer is returned. - **int uri\_realloc(uri\_t\* object, char\* uri, int uri\_length)** The **uri** is splitted into fields in the previously allocated **object** structure. The URI is put in cannonical form and **URI\_CANNONICAL is returned.** If it cannot be put in cannonical form, nothing is done and **URI\_NOT\_CANNONICAL** is returned. - **void uri\_free(uri\_t\* object)** The **object** previously allocated by **uri\_alloc** is deallocated. - **uri\_t\* uri\_abs(uri\_t\* base, char\* relative\_string, int relative\_length)** Transform the relative URI **relative\_string** into an absolute URI using **base** as the base URI. The returned **uri\_t** object is allocated statically and must not be freed. - **uri\_abs\_1(uri\_t\* base, uri\_t\* relative)** Transform the relative URI **relative** into an absolute URI using **base** as the base URI. The returned **uri\_t** object is allocated statically and must not be freed. - **int uri\_info(uri\_t\* object)** returns the content of the **info** field. - **char\* uri\_scheme(uri\_t\* object)** returns the content of the **scheme** field. - **char\* uri\_host(uri\_t\* object)** returns the content of the **host** field. - **char\* uri\_port(uri\_t\* object)** returns the value of the **port** field of the **object.** If the **port** field is empty, returns the default port for the corresponding scheme. For instance, if the scheme is **http** the **80** string is returned. The returned string is statically allocated and must not be freed. - **char\* uri\_path(uri\_t\* object)** returns the content of the **path** field. - **char\* uri\_params(uri\_t\* object)** returns the content of the **params** field. - **char\* uri\_query(uri\_t\* object)** returns the content of the **path** field. - **char\* uri\_frag(uri\_t\* object)** returns the content of the **frag** field. - **char\* uri\_user(uri\_t\* object)** returns the content of the **user** field. - **char\* uri\_passwd(uri\_t\* object)** returns the content of the **passwd** field. - **char\* uri\_netloc(uri\_t\* object)** returns a concatenation of the **host** and **port** field, separated by a **:.** If the **host** field is not set, the null pointer is returned and a message is printed on stderr. The returned string is statically allocated and must not be freed. - **char\* uri\_auth\_netloc(uri\_t\* object)** returns a concatenation of the **host** and **port** field, separated by a **:.** If the **user** field is set, the **user** and **passwd** fields are prepended to the netloc, separated by a @. If the **host** field is not set, the null pointer is returned and error condition is set. The returned string is statically allocated and must not be freed. - **char\* uri\_auth(uri\_t\* object)** returns a concatenation of the **user** and **passwd** field, separated by a **:** or an empty string if any of them is not set. The returned string is statically allocated and must not be freed. - **char\* uri\_all\_path(uri\_t\* object)** returns a concatenation of the **path, params** and **query** fields in the form **/path;params?query. Note that a leading slash is only prepended** to the returned value if the object is **not** a relative URI. The returned string is statically allocated and must not be freed. - **void uri\_info\_set(uri\_t\* object, int value)** set the info field to value. - **void uri\_scheme\_set(uri\_t\* object, char\* value)** set the scheme field to value. The **URI\_INFO\_RELATIVE** is updated according to the new value. - **void uri\_host\_set(uri\_t\* object, char\* value)** set the host field to value. The **URI\_INFO\_RELATIVE** is updated according to the new value. - **void uri\_params\_set(uri\_t\* object, char\* value)** set the params field to value. - **void uri\_query\_set(uri\_t\* object, char\* value)** set the query field to value. - **void uri\_user\_set(uri\_t\* object, char\* value)** set the user field to value. - **void uri\_passwd\_set(uri\_t\* object, char\* value)** set the passwd field to value. - **void uri\_copy(uri\_t\* to, uri\_t\* from)** copy the content of object **from** into object **to.** - **uri\_t\* uri\_clone(uri\_t\* from)** creates a new object containing the same data as **from.** The returned object must be freed using uri\_free. - **void uri\_clear(uri\_t\* object)** clear all information contained in **object.** - **void uri\_set\_root(const char\* root)** Set the path that **uri\_furi** will prepend to the FURI. By default it is the empty string. - **const char\* uri\_get\_root()** Get the path set by **uri\_set\_root** or empty string. - **char\* uri\_furi(uri\_t\* object)** returns a string containing the FURI (File equivalent of an URI) built from **object.** The returned string is statically allocated and must not be freed. - **char\* uri\_uri(uri\_t\* object)** returns a string containing the URI built from **object.** The returned string is statically allocated and must not be freed. - **void uri\_string(uri\_t\* object, char\*\* stringp, int\* string\_sizep, int flags)** Build a string representation of **object** in **stringp** according to **flags.** Possible values of flags is described in the **uri\_cannonicalize\_string** function. Upon return the **stringp** pointer points to a static array of **stringp\_size** bytes allocated with malloc. If **stringp** is not null it must point to a buffer allocated with malloc and is reallocated to fit the needs of the string conversion. This function is the backend of all object to string translation functions. - **char\* uri\_escape(char\* string, char\* range)** return a statically allocated copy of **string** with all characters found in the the **range** string transformed in escaped form (%xx). A few examples of **range** argument are defined: URI\_ESCAPE\_RESERVED, URI\_ESCAPE\_PATH, URI\_ESCAPE\_QUERY, and uri\_escape\_unsafe. - **char\* uri\_unescape(char\* string)** return a statically allocated copy of **string** with all escape sequences (%xx) transformed to characters. - **char\* uri\_cannonicalize\_string(char\* uri, int uri\_length, int flag)** returns the cannonical form of the **uri** given in argument. The cannonical form is formatted according to the value of **flag.** Values of flag are bits that can be ored together. **URI\_STRING\_FURI\_STYLE** return a FURI, **URI\_STRING\_URI\_STYLE** return an URI, **URI\_STRING\_ROBOTS\_STYLE** return the corresponding robots.txt URI, **URI\_STRING\_URI\_NOHASH\_STYLE** do not include the frag in the returned string. Returns 0 if **uri** is malformed. - **uri\_t\* uri\_cannonical(uri\_t\* object)** returns an object containing the cannonical form of **object.** If the **URI\_MODE\_CANNONICAL** flag is set, the object itself is returned. - **int uri\_consistent(uri\_t\* object)** Returns 0 if **object** contains unparsable URL, returns != 0 if **object** contains a well formed URL. Must be called after a set of field changes to reset flags and ensure that modified URL is well formed. ## HTTP FUNCTIONS - **char\* uri\_robots(uri\_t\* object)** returns a string containing the URI of the robots.txt file corresponding to the URI contained in object. For instance, if the URI contained in object is **http://www.foo.com/dir/dir/file.html** the returned string will be **http://www.foo.com/robots.txt.** The returned string is statically allocated and must not be freed. ## CANNONICAL FORM The cannonical form of an URI is an arbitrary choice to code all the possible variations of the same URI in one string. For instance http://www.foo.com/abc"def.html will be transformed to http://www.foo.com/abc%22def.html. Most of the transformations follow the instructions found in **draft-fielding-uri-syntax-04** but some of them don't. Additionally, when the path of the URI contains dots and double dots, it is reduced. For instance **http://www.foo.com/dir/.././file.html** will be transformed to **http://www.foo.com/file.html.** If the **URI\_MODE\_CANNONICAL** flag is set, the **uri\_t** object always contains the cannonical form of the URL. The original form is lost. If the **URI\_MODE\_CANNONICAL** flag is not set, the cannonical form of the URI is stored in a separate object. The **uri\_t** object contains the original form of the URI. It takes more memory to store but may be usefull in some situations. ## ERROR HANDLING When an error occurs (URI cannot be cannonicalized or parsed, for instance), the global variable **uri\_errstr** contains the full text of the error message. This variable is never reset by the library functions if no error occurs. Additionally, the error string may be printed on the error chanel (STDERR) if the **URI\_MODE\_ERROR\_STDERR** flag is set. This is the default. ## STRICTNESS The draft describing URI syntax (draft-fielding-uri-syntax-04) specifies that an URI of the type http:g may be interpreted in two different ways. If the **URI\_MODE\_URI\_STRICT** flag is set, the library interprets it as an absolute URI, otherwise it is a relative URI. If the **URI\_MODE\_URI\_STRICT** is not set, the **URI\_MODE\_URI\_STRICT\_SCHEME** may be set so that a relative URI containing a scheme is interpreted as an absolute URI only if the scheme is different from the scheme of the base URI. ## FURI It is sometimes convinient to convert an URI into a path name. Some functions of the **uri** library provide such a conversion (uri\_furi for instance). These path names are called FURI (File equivalent of an URI) for short. Here is a description of the transformation. ``` http://www.ina.fr:700/imagina/index.html#queau | \____________/ \________________/\____/ | | | lost | | | | | | / | | | | | | | | | | | / | | | /^^^^^^^^^^^^^\/^^^^^^^^^^^^^^^^\ http/www.ina.fr:700/imagina/index.html ``` ## EXAMPLES **Show cannonical form of URI** ``` char* uri = "http://www.foo.com/"; uri = uri_cannonicalize_string(uri, strlen(uri), URI_STRING_URI_STYLE); if(uri) printf("uri = %s\n", uri); ``` **Show the host and port of URI (netloc)** ``` char* uri = "http://www.foo.com:7000/"; uri_t* uri_object = uri_object(uri, strlen(uri)); if(uri_object) printf("netloc = %s\n", uri_netloc(uri_object)); ``` **Change the query part of URI and show it** ``` char* uri = "http://www.foo.com/cgi-bin/bar?param=1"; uri_t* uri_object = uri_object(uri, strlen(uri)); if(uri_object) { uri_query_set(uri_object, "param=2"); printf("uri = %s\n", uri_uri(uri_object)); } ``` ## ADDING NEW SCHEMES Add the name of the scheme in the SCHEMES file. If nothing else this will bind the scheme to a generic parser following the URI parsing rules. If you want to define specific behaviour for this scheme, mimic the uri\_scheme\_http.c file and recompile. If [gperf(1)](/gperf/1.md) complains because it has conflicts you'll have to play with the -k option in order to find a working range that does not conflict and takes a few space as possible. ## AUTHOR Loic Dachary loic@senga.org ## SEE ALSO draft-fielding-uri-syntax-04