We've expanded our Linux news coverage and improved our search! Search for all things Linux across O'Reilly!

Simplify Network Programming with libCURL
Pages: 1, 2

FTP Upload (Push a File to a Legacy System)

Legacy system uploads and downloads go hand in hand. The stub program step3 uses libCURL to log in to a remote FTP host and upload a file. It also describes a way to use a true C++ object as the callback handler (albeit indirectly).

step2 takes the remote FTP host, log in, and password as arguments. (A real app would read in this data from a config file; for now, pretend it's not a security problem to specify it on the command line.) The example merges the hostname with the target file name to form the URL:

ftp://host/file

It also concatenates the log in and password into a string used as the context option CURLOPT_USERPWD:

login:password

Note that you can also put the log in info in the URL:

ftp://login:password@host/file

This URL format is simply another means to pass the log in information to the API. Unlike a browser, libCURL doesn't use a cache or history bar that prying eyes can later discover. (Hopefully, the remote FTP server doesn't log that kind of information, either.) Nor is the information available via ps browsing.

For firewall-friendly transfers, the context option CURLOPT_FTP_USE_EPSV tells the library to use passive FTP.

libCURL doesn't limit you to file transfers alone. You can also send arbitrary FTP commands, such as mkdir or cwd. Store the commands in a libCURL linked list (curl_slist*):

struct curl_slist* commands = NULL ;
commands = curl_slist_append( commands , "mkdir /some/path" ) ;
commands = curl_slist_append( commands , "mkdir /another/path" ) ;
... 

The CURLOPT_QUOTE context option executes commands after logging in to the FTP server but before transferring data. CURLOPT_POSTQUOTE specifies a list of commands to execute after having transferred data.

curl_easy_setopt( ctx , CURLOPT_QUOTE , commands ) ;
// ... call curl_easy_perform() to run the FTP session ...
curl_slist_free_all( commands ) ;

You can use these context options to curl-ify your old FTP scripts. step3 uses CURLOPT_QUOTE to call cwd / such that the file uploads relative to the root directory of the FTP server. (Without an explicit directory change, uploads operate to a path relative to the user's home directory.)

The context option CURLOPT_UPLOAD tells the library this will be an upload call.

CURLOPT_FTPAPPEND tells libCURL to append to the target file instead of overwriting it. It's not necessary in this example, but it's something you often see in legacy FTP jobs.

Similar to downloading data, when uploading you have a choice between passing the libCURL library a file handle or creating the data yourself in a callback.

To upload data from an existing file handle, set that FILE* as the context option CURLOPT_READDATA.

To use a callback instead (for example, to generate upload data on the fly), assign a function to the context option CURLOPT_READFUNCTION. The function signature is very similar to that of CURLOPT_WRITEFUNCTION:

size_t function(
  char* buffer ,
  size_t size ,
  size_t nitems ,
  void* userData
) ;

The difference is that buffer is where you store data in this case, and the product size*nmemb is the maximum number of bytes you can put there. (Return the number of bytes you put in the buffer.) userData is the value assigned to CURLOPT_READDATA.

step3's callback function is rather brief. I employ a C++ template technique to use an object indirectly as a callback handler. If you're not familiar with templates, note that the declaration

template< typename T > 
class UploadHandler {
  ...

means the class UploadHandler is incomplete as written. The data type T comes from elsewhere in the code, when registering the function with libCURL:

curl_easy_setopt(
  ctx ,
  CURLOPT_READFUNCTION ,
  UploadHandler< UploadData >::execute
);

Here, UploadData is the type of object the handler function will use to do the work.

In turn, the static class function UploadHandler::execute() is a mere pass-through: it casts the userData value to type T and invokes T::execute() to do the actual work.

static size_t execute(
  char* buffer ,
  size_t size ,
  size_t nitems ,
  void* userData
){

  T* realHandler = static_cast< T* >( userData ) ;
  return( realHandler->execute( buffer , size , nitems ) ) ;

}

UploadHandler will work with any class that implements a fitting execute member function. I could have used standard inheritance instead:

// ... inside UploadHandler::execute() ...

Handler* h = static_cast< Handler* >( userData ) ;
return( h->execute( ... ) ) ;

I prefer the flexibility of templates, though. Inheritance would tie all handler objects to the Handler interface.

Similar to download callbacks, the library may call upload callbacks called several times for a single file. Code accordingly.

HTTP POST (Populate a Web Form)

HTTP POST operations send form data to a web server as well as making code-to-code calls such as those in web services. The request body comprises the POST data.

This article's final example, step4, demonstrates how to use libCURL for an HTTP POST. It also explains how to set up custom HTTP request headers, such as browser identification.

The POST body is just a string with & characters between key=value pairs:

const char* postData = "param1=value1&param2=value2&..." ;

Pass this string to the library by assigning it to the CURLOPT_POSTFIELDS option:

curl_easy_setopt( ctx , CURLOPT_POSTFIELDS , postData ) ;

Assign a curl_slist* to CURLOPT_HTTPHEADER to set custom HTTP headers:

curl_slist* responseHeaders = NULL ;

responseHeaders = curl_slist_append(
  responseHeaders ,
  "Expect: 100-continue"
) ;

// ... other curl_slist_append() calls ...

curl_easy_setopt(
  ctx ,
  CURLOPT_HTTPHEADER ,
  responseHeaders
) ;

Note that libCURL clients skip the intermediate step of downloading and processing a form's HTML. In turn, it is unaware of any hidden fields or client-side technologies used therein (such as JavaScript). Put another way, you have to know what fields the web server expects before you can use a libCURL client to POST data.

Conclusion

libCURL provides clean, simple networking for your native-code applications. With this API in your toolbox, you can incorporate one-off FTP operations into your main application, automate HTTP POST requests, and more.

There's much more to libCURL than I've presented here. The examples should, however, give you a head start in putting libCURL to use in your own apps.

Resources

• The article's sample code includes the source for the stub programs, as well as a JSP and PHP page with which to test step4. (The JSP requires a servlet spec 2.4 container, such as Tomcat 5, and a proper 2.4 web.xml.)

  The pages simply echo the request headers and POST parameters received from the client.

• The curl web site has documentation and tutorials.

• The TCPMon utility ships with Apache's Axis web services project. It's a listening proxy that shows client/server conversations in a GUI window. I've found it invaluable for debugging problems with my curl code, especially HTTP POST operations.

  TCPMon is a Java application and is thus portable to any Java-enabled platform that meets the JDK version requirements.

• libCURL is especially useful for creating REST-based web services clients. Also known as XML over HTTP, REST web service calls encapsulate HTTP GET or POST requests instead of wrapping them in a SOAP envelope. Amazon.com, for example, offers its public web services API via REST as well as SOAP. Yahoo's web services use REST exclusively.

Ethan McCallum grew from curious child to curious adult, turning his passion for technology into a career.

Return to the Linux DevCenter.