Dr. Gaurav Khanna is a brilliant Astrophysicist from the University of Massachusetts Dartmouth who works on different problems in theoretical and computational astrophysics. To assist him on some of his processor intensive work, he has built a supercomputing cluster out of 16 PS3’s called the gravity PS3 Gravity Grid. He is a pioneer in the PS3 supercomputing field, and his shared his knowledge on building clusters at ps3cluster.org.
I had a chance to interview Dr. Khanna, and we talked about how this all got started, cluster speed and his thoughts on the lack of OtherOS support in the revamped PS3 line. Immense thanks to Dr. Khanna for being extremely personable, easy to talk to and for answering all of our questions 🙂
Being one of the pioneers in this PS3 cluster field, how do you think using the PS3 in this way has changed the landscape of research?
The PS3 has definitely significantly impacted various areas of computation-based research. Perhaps the best example of this is Stanford’s Folding@Home project that is nearing 10 petaflops, of which PS3s contribute an important fraction. And then there are dozens of small clusters at various universities like mine.
In my case in particular, I have been able to do calculations that I simply couldn’t do before. And that makes the whole approach invaluable to me, of course. But beyond what I have been able to do, the thought that one should seriously evaluate commodity gaming hardware (consoles and graphics cards) for high-performance scientific computing is leading towards a transformation of the supercomputing industry.
Prior to being able to utilize the PS3, how you were able to accomplish your research?
Academic researchers in the US have access to supercomputing facilities through various federal agencies (such as the NSF, NASA etc). To obtain supercomputing time, one submits a proposal to the federal agency, which is peer-reviewed and prioritized. Alternatively, one can rent supercomputers at the rate of $1 per CPU per hour. To give you an idea, a single black hole research simulation of interest to me, can take 5,000 — 10,000 CPU-hours. And to explore a problem in depth, one would want to do several dozen such simulations, at the very least!
Yes, I have used such facilities before for my black hole astrophysics research and continue to do so. They are an awesome resource and are managed very well. But, since they are heavily shared facilities they often have long queues and strong restrictions on their usage. I realized that for a specific, computationally intensive, black hole research problem that I was working on, I simply did not have the supercomputing time available to me for a thorough study. It was mainly for this reason, I started looking into “creative” solutions.
It’s an amazing concept to use the PS3 in a cluster the way you are. Can you walk us through how you first got involved with Sony on this project and how it came to fruition Was it difficult to “cell” this idea?
Thanks. Well before the PS3 was released in 2006, IBM and Sony had already released a lot of information on the Cell BE — its exceptionally strong performance and incredible potential. I have never had an interest in gaming, but I happened to come across an article about the Cell BE in 2005 and started wondering about the possibilities of using the PS3 for my computational research. When the PS3 was released in late 2006 and it was made available as an “open platform” there was no reason to not get one immediately and actually try things out — except for the long lines at Best Buy, of course! My wife braved those long lines and shortages and actually managed get me one for Christmas. You can imagine how I spent that winter break; and no, its wasn’t spent playing Motorstorm!
Once I realized the potential of Cell BE by experimenting with it on my own research codes, I realized that I would only need a small sized PS3 cluster to be able to complete the computationally intensive simulations that I was having trouble doing otherwise. Envisioning that I would be laughed at if I asked a research funding agency to buy me a few gaming consoles, I decided to try my luck directly with Sony, instead.
Talking to Sony about a proposal like this one wasn’t easy. Its very easy to contact a Sony representative and talk; but explaining what I really needed was extremely difficult. I believe the biggest challenge was getting the right person within Sony to appreciate what I wanted to do. And Sony is a BIG company. After trying for many months, I managed to get that right person, who happened to be in Sony R&D. Once that happened, it took no time at all. I was amazed by their responsiveness and their interest. I am still in touch with many of the people within Sony that helped me along the way and pointed me in the right direction. Without their help and interest, my cluster wouldn’t have happened.
How fast is this PS3 Cluster?
I currently have a cluster with 16 PS3s. On my research codes, my cluster performs comparably to a cluster of approximately 100 cores of a high-end Intel Xeon processor. What is particularly interesting is not just the overall performance, but its extremely low cost. It is about ten (10) times more cost effective than a traditional compute cluster. Moreover, it is a dedicated resource. It is not shared, there are no restrictions, and I don’t have to write proposals and get reviewed etc. or pay money to make use of it :-). All these things help enhance my research productivity significantly.
How many years or months do you think you’re research has been accelerated by using the PS3 cluster?
As I mentioned before, without it I actually wouldn’t have been able to do some of the simulations I needed — simply because of the available supercomputing time I had and due to some of the restrictions involved. So, to me it has been an indispensable resource.
I have been able to publish four (4) research articles based on the data generated by the cluster, and expect many more to come in the future 🙂
What are your thoughts on the lack of OtherOs support in the latest version of the PS3?
I am simply devastated by this decision. I understand that the decision was made to cut developmental cost. I think it is extremely unfortunate that Sony had to do this.
Given the $299 price point, do you think more researchers would have been able to expand their work if OtherOs support had not been removed?
Absolutely! The price drop made the cost effectiveness of a PS3 cluster even higher (over a factor of 10) when compared to traditional compute clusters, and the Slim model consumes significantly less power as well. I would have immediately looked into increasing my PS3 cluster size (if I hadn’t run out of room in my lab!).
Lastly, do you think it would be wise for Sony to consider restoring OtherOs support, even if just on a 1 to 1 basis?
Without a doubt! I think there are a number of options that Sony will always have to restore the OtherOS support. Short of simply reversing their decision, they could sell a PS3 Linux Kit at the price of a game perhaps? Of course, there are many other viable options. I am extremely hopeful that PS3 Linux will survive. I’m keeping my fingers crossed 🙂
Again, many thanks to Dr. Khanna for his time. I actually felt smarter just talking to him 🙂
