About the Book

This book shows how to access the web using a Java program rather than a Web browser. There is a tremendous amount of information available on the Web. From current weather, sock prices, or the location of a package that you shipped, a bot can access all of this information. This book begins by showing how to create simple bots that will retrieve information from a single website. Then a spider is developed that can move from site to site as it crawls across the Web. Next we build aggregators that can take data from many sites and present a consolidated view. As the chapters progress a reusable package is developed that you can use in your own programs. The book concludes showing how bots of the future will work with SOAP.

Whats New

Newest version: Version 1.4 of the Bot Package.

Information From the Book

• Errata - A list of errors from the book.
• Sample Chapter - See Chapter 1
• Book's Index - What is in the book
• FAQ - About/Frequently asked questions (FAQ)
• Updated Material - Updated material from the book, such as the bot package(bot.jar) and examples
• Java Link Checker - A simple LGPL(free) spider that I wrote to check links. Does not use the bot package, totally self contained. Does not support all of the advanced features of the bot package used in my book.
• Java Artificial Intelligence - My online book about programming neural networks in Java, over 300 pages
• Java Articles - Java articles written by me

Reviews

Here are some reviews of my spider programming book.

• Java Pro 9/2002 - Although this book is a bit pricey for most, it may well contain the best collection of code and explanations available for this category of applications, particularly for Java developers.
• Focus on Java - But, this book pulls together in a single volume everything you need to get started writing your own agents in Java. (Want to check eBay automatically to find that router bargain?) I, especially, appreciate this book, because it is more practical and less theoretical than some of its rivals.

ScriptSearch.com

HTTP Get/Post in C++

This program is a simple utility that is used to GET/POST from a website. It is based on an earlier version that appeared in Windows Developer Journal in January 2001. I've made a few improvements to it based on reader suggestions. Mainly it has been transformed to a C++ file. It now compiles just find with C++. A small buffer over-run has also been fixed.


[Click to download Source Code(C/C++)]
[Click to download Compiled EXE(WIN32)]