Toolbox

Our Emperor Sponsors

  • InternetNZ
  • Google
  • IBM
  • HP

<-- Back to schedule

Gearman: Map/Reduce and Queues for everyone!

Time:13:30 - 14:15
Day:Thursday 21 January 2010
Location:Main Auditorium (MFC)
Project: Gearman

Many people today view topics like Map/Reduce and queue systems as advanced concepts that require in-depth knowledge and time consuming software setup to get started. Gearman is changing all that by making this barrier to entry as low as possible with an open source, distributed job queuing system. Users are only a package and command line tool away from having their own distributed system installed to help with a variety of tasks like log aggregation, log analyzation, remote management, and building scalable websites.

Gearman was orignially designed at Danga to help scale LiveJournal.com, but now Gearman development is very active again with a re-write in C, additional features like persistent queues, and new adoption happening within many organizations. We’re also introducing new and improved client APIs for many languages, including new user-defined functions for Drizzle, MySQL, and PostgreSQL.

The Gearman APIs and simple server installation allow you to have your own distributed computing framework setup in a matter of minutes, allowing you to focus on your own code rather than the plumbing. It is trivial to integrate Gearman with existing applications, and there are even command line interfaces so everyday shell hackers can make use of Gearman! This session will introduce these new concepts, show how to get started, and then describe a few common use cases.

Eric Day

Eric Day works full-time on Drizzle and Gearman at Sun Microsystems. He has been writing high-performance, multi-threaded servers and databases for most of his career, always with an emphasis on efficiency and security. A majority of that time was at Concentric where he wrote complete HTTP/SSL, DNS, and IMAP implementations, along with designing and implementing custom storage and database systems. Most of his work has been done in clustered and distributed environments. When not hacking on code, he can be found running, biking, or enjoying a good vegan meal.