ibrowse is a HTTP client. The following are a list of features. - RFC2616 compliant (AFAIK) - supports GET, POST, OPTIONS, HEAD, PUT, DELETE, TRACE, MKCOL, PROPFIND, PROPPATCH, LOCK, UNLOCK, MOVE and COPY - Understands HTTP/0.9, HTTP/1.0 and HTTP/1.1 - Understands chunked encoding - Can generate requests using Chunked Transfer-Encoding - Pools of connections to each webserver - Pipelining support - Download to file - Asynchronous requests. Responses are streamed to a process - Basic authentication - Supports proxy authentication - Can talk to Secure webservers using SSL - any other features in the code not listed here :)

ibrowse is available under two different licenses. LGPL and the BSD license.

Comments to : Chandrashekhar.Mullaparthi@gmail.com

Version : 1.5.6

Latest version : git://github.com/cmullaparthi/ibrowse.git

CONTRIBUTIONS & CHANGE HISTORY

08-11-2009 - * Added option headers_as_is

04-10-2009 - * Patch from Kostis Sagonas to cleanup some code and suppress dialyzer warnings

24-09-2009 - * When a filename was supplied with the 'save_response_to_file' option, the option was being ignored. Bug report from Adam Kocoloski

05-09-2009 - * Introduced option to allow caller to set socket options.

29-07-2009 - * The ETS table created for load balancing of requests was not being deleted which led to the node not being able to create any more ETS tables if queries were made to many number of webservers. ibrowse now deletes the ETS table it creates once the last connection to a webserver is dropped. Reported by Seth Falcon. * Spurious data being returned at end of body in certain cases of chunked encoded responses from the server. Reported by Chris Newcombe.

03-07-2009 - Added option {stream_to, {Pid, once}} which allows the caller to control when it wants to receive more data. If this option is used, the call ibrowse:stream_next(Req_id) should be used to get more data. - Patch submitted by Steve Vinoski to remove compiler warnings about the use of obsolete guards

29-06-2009 - * Fixed following issues reported by Oscar Hellström - Use {active, once} instead of {active, true} - Fix 'dodgy' timeout handling - Use binaries internally instead of lists to reduce memory consumption on 64 bit platforms. The default response format is still 'list' to maintain backwards compatibility. Use the option {response_format, binary} to get responses as binaries. * Fixed chunking bug (reported by Adam Kocoloski) * Added new option {inactivity_timeout, Milliseconds} to timeout requests if no data is received on the link for the specified interval. Useful when responses are large and links are flaky. * Added ibrowse:all_trace_off/0 to turn off all tracing * Change to the way responses to asynchronous requests are returned. The following messages have been removed. * {ibrowse_async_response, Req_id, {chunk_start, Chunk_size}} * {ibrowse_async_response, Req_id, chunk_end} * Fixed Makefiles as part of Debian packaging (thanks to Thomas Lindgren) * Moved repository from Sourceforge to Github

11-06-2009 - * Added option to control size of streamed chunks. Also added option for the client to receive responses in binary format.

21-05-2008 - * Fixed bug in reading some options from the ibrowse.conf file. Reported by Erik Reitsma on the erlyaws mailing list * Fixed bug when cleaning up closing connections

27-03-2008 - * Major rewrite of the load balancing feature. Additional module, ibrowse_lb.erl, introduced to achieve this. * Can now get a handle to a connection process which is not part of the load balancing pool. Useful when an application is making requests to a webserver which are time consuming (such as uploading a large file). Such requests can be put on a separate connection, and all other smaller/quicker requests can use the load balancing pool. See ibrowse:spawn_worker_process/2 and ibrowse:spawn_link_worker_process/2 * Ram Krishnan sent a patch to enable a client to send a lot of data in a request by providing a fun which is invoked by the connection handling process. This fun can fetch the data from any where. This is useful when trying to upload a large file to a webserver. * Use the TCP_NODELAY option on every socket by default * Rudimentary support for load testing of ibrowse. Undocumented, but see ibrowse_test:load_test/3. Use the source, Luke! * New function ibrowse:show_dest_status/2 to view state of connections/pipelines to a web server

20-02-2008 - Ram Krishnan sent another patch for another hidden bug in the save_response_to_file feature.

07-02-2008 - Ram Krishnan (kriyative at gmail dot com) sent a simple patch to enable specifying the filename in the save_response_to_file option. When testing the patch, I realised that my original implementation of this feature was quite flaky and a lot of corner cases were not covered. Fixed all of them. Thanks Ram!

17-10-2007 - Matthew Reilly (matthew dot reilly at sipphone dot com) sent a bug report and a fix. If the chunk trailer spans two TCP packets, then ibrowse fails to recognise that the chunked transfer has ended.

29-08-2007 - Bug report by Peter Kristensen(ptx at daimi dot au dot dk). ibrowse crashes when the webserver returns just the Status line and nothing else.

28-06-2007 - Added host_header option to enable connection to secure sites via stunnel

20-04-2007 - Geoff Cant sent a patch to remove URL encoding for digits in ibrowse_lib:url_encode/1. ibrowse had a dependency on the inets application because the ibrowse_http_client.erl invoked httpd_util:encode_base64/1. This dependency is now removed and the encode_base64/1 has been implemented in ibrowse_lib.erl

06-03-2007 - Eric Merritt sent a patch to support WebDAV requests.

12-01-2007 - Derek Upham sent in a bug fix. The reset_state function was not behaving correctly when the transfer encoding was not chunked.

13-11-2006 - Youns Hafri reported a bug where ibrowse was not returning the temporary filename when the server was closing the connection after sending the data (as in HTTP/1.0). Released ibrowse under the BSD license

12-10-2006 - Chris Newcombe reported bug in dealing with requests where no body is expected in the response. The first request would succeed and the next request would hang.

24-May-2006 - Sean Hinde reported a bug. Async responses with pipelining was returning the wrong result.

08-Dec-2005 - Richard Cameron (camster@citeulike.org). Patch to ibrowse to prevent port number being included in the Host header when port 80 is intended.

22-Nov-2005 - Added ability to generate requests using the Chunked Transfer-Encoding.

08-May-2005 - Youns Hafri made a CRUX LINUX port of ibrowse. http://yhafri.club.fr/crux/index.html

Here are some usage examples. Enjoy!

5> ibrowse:start(). {ok,<0.94.0>}

%% A simple GET 6> ibrowse:send_req("http://intranet/messenger/", [], get). {ok,"200", [{"Server","Microsoft-IIS/5.0"}, {"Content-Location","http://intranet/messenger/index.html"}, {"Date","Fri, 17 Dec 2004 15:16:19 GMT"}, {"Content-Type","text/html"}, {"Accept-Ranges","bytes"}, {"Last-Modified","Fri, 17 Dec 2004 08:38:21 GMT"}, {"Etag",""aa7c9dc313e4c41:d77""}, {"Content-Length","953"}], "<html>\r\n\r\n<head>\r\n<title>Messenger</title>\r\n<meta name="GENERATOR" content="Microsoft FrontPage 5.0">\r\n<meta name="ProgId" content="FrontPage.Editor.Document">\r\n<meta name="description" content="Messenger Home Page">\r\n</head>\r\n\r\n<frameset border="0" frameborder="0" rows="60,">\r\n <frame src="/messenger/images/topnav.html" name="mFrameTopNav" scrolling="NO" target="mFrameMain">\r\n <frameset cols="18%,">\r\n <frameset rows="*,120">\r\n <frame src="index-toc.html" name="mFrameTOC" target="mFrameMain" scrolling="auto" noresize="true">\r\n <frame src="/shared/search/namesearch.html" name="mFrameNameSearch" scrolling="NO" target="mFrameMain">\r\n </frameset>\r\n <frame src="home/16-12-04-xmascardsmms.htm" name="mFrameMain" scrolling="auto" target="mFrameMain" id="mFrameMain">\r\n </frameset>\r\n <noframes>\r\n <body>\r\n\r\n <p><i>This site requires a browser that can view frames.</i></p>\r\n\r\n </body>\r\n </noframes>\r\n</frameset>\r\n\r\n</html>"}

%% ============================================================================= %% A GET using a proxy 7> ibrowse:send_req("http://www.google.com/", [], get, [], [{proxy_user, "XXXXX"}, {proxy_password, "XXXXX"}, {proxy_host, "proxy"}, {proxy_port, 8080}], 1000). {ok,"302", [{"Date","Fri, 17 Dec 2004 15:22:56 GMT"}, {