Friday, April 30, 2010

Farm - process, store and simplify data access

Alright, so making a computing farm that embraces in its fold, wired desktop PCs, wireless laptops, and even non-x86 architectures such as CUDA acceleration and RISC-based ARM processor smartphones...   does work. For the curious, here are some geeky specs (of the farm overall) to get excited about:

19 logical processing units (the majority clocked over 2GHz, this number does not take the phones/GPUs into consideration)
22 GigaBytes (GB) of RAM
9.5 TeraBytes (TB) of secondary storage
OS: Windows-only, except for the Symbian-based phones that were thrown in.

What were the challenges?
 - Connectivity! Why the heck don't we have multi-Gigabit wireless interfaces yet?
 - Syncing and bunching them into "similar" process groups, to ensure chunks are "checked in" in the right sequence.
 - Keeping the farm fed continuously with data to crunch, else it ends up being one heck of a mean parallel processing entity simply waiting at a faster speed and in more numbers to boot (since it isn't comprised of a single "PC" but is a collection) !

No comments:

Post a Comment

No spam please, my eyes are watching you!