HTTP Toolbox

Most of us regularly interact with the HTTP protocol at some level. These days, it is very common to interact with it by tapping or clicking on buttons and icons presented in the user interface of an app.

Several years ago, even the most casual internet user would have been familiar with finding a web page by typing a URL into a browser like Chrome, Firefox, or Safari. From there, he could continue his usage of HTTP by clicking through “clickable” text and images presented on the web page.

Over time, the interfaces between HTTP and non-developers have evolved. There are an ever increasing number of ways to interact with HTTP for non-developers.

It is also true that there are a lot of different ways to interact with HTTP for developers. In this post, we will look at a couple of different ways developers transfer data from one server to another over HTTP. More specifically, we will look at different options for HTTP clients.

Using cURL

cURL is a handy tool to have in your developer toolbox. We will use the command line tool here, but the cURL project also gives you access to the libcurl library that powers the command line tool.

First, you should check to see if you have the command line tool installed on your machine by attempting to run any curl command:

$ curl

If you get an error message, you will need to install the curl package on your machine.

Here is a resource for installing cURL on different Linux distributions. This site focuses on using Linux based systems, but you can find versions of cURL that work for other operating systems as well.

Now, try running the curl command again:

$ curl

You should see output that looks similar to the following message now:

cURL allows you to run commands with a variety of options. It is also capable of using different network protocols, but it will assume HTTP as the default. Let’s run a basic command to request the resource located on the example.com server at path /.

$ curl example.com/

As you can see, the curl tool printed out the payload returned in the HTTP response from the example.com server.

How do we know that the curl tool made an HTTP request? Let’s run our basic command one more time, but include the -I flag.

$ curl -I example.com/

Using the -I option commands curl to only display the headers for the response it gets from the remote server. The headers output also includes the start line of the response message:

As you can see, the example.com server responded to our request using version 1.1 of HTTP.

The benefit of using the cURL program is that it is a powerful tool. In fact, its network engine is used in libraries for other programming languages. Also, since it is a command line tool, you can easily use it in bash scripts.

One trade off is that you have less control over the interface of the tool. You have to adhere to the standards set by the cURL developers for input and output data.

Building Your Own Program

You can, of course, build your own tool that transfers data over HTTP. Let’s use the Ruby programming language to write a basic program that transfers data from the example.com server using HTTP.

# http.rb

require 'net/http'

response = Net::HTTP.get('example.com', '/')
puts response

Here, we make use of the net/http library that comes with the Ruby language in order to exchange data over HTTP. Running this program will make an HTTP request to the example.com server and print the payload of the HTTP response to the console.

The benefit of writing your own HTTP client is that you have more control over building the request and processing the response. In this case, we just needed to print the response payload to the console.

By using an HTTP library for a particular programming language, you can seamlessly integrate HTTP communication into other code.

The trade off is that you are responsible for taking care of all of the details. So you have to understand more about how HTTP works. But that should not be a problem because we will continue to learn about the HTTP protocol.

Using the Browser

Finally, we’ll address the HTTP client known as the browser. Using a browser is so fundamental that it may seem unnecessary to even bring it up in this discussion about different ways developers interact with the HTTP protocol.

It is for that very reason that we bring up the browser in this article. Because the browser is something many of us used way before becoming software developers, it can be easy to forget that it’s just another tool that enables the transfer of data from one machine to another over HTTP. In that sense, it is just like cURL or the simple ruby program we wrote.

Granted, the browser is a powerful tool with many more features than other tools. Nevertheless, it has its benefits and trade offs as well.

One major benefit the browser offers as an HTTP client is that it makes usage of the World Wide Web very user friendly. For example, its ability to render a rich visual experience makes navigating the web easier.

However, there are situations in which a browser is not ideal. Browsers are very opinionated about how to handle HTTP communication. They make choices about how to handle data that you cannot control. If you want to create a more custom HTTP request with special configurations, you may need one of the other tools discussed above.

Conclusion

In software engineering, there are often many options for handling certain tasks. Handling the transfer of data over HTTP is no different. HTTP clients vary in the number of features they come with out of the box and in their simplicity of use. When you need to decide which client to use, it is helpful know the pros and cons of the various options in your toolbox.