Sometimes, the most exciting opportunities are where you’re able to bring past experiences to bear on something new. And when you have the chance to join a team of great people to solve really big problems, that makes it even better. That's why I'm so thrilled to be joining Anyscale.

I first got to know about Anyscale and Ray a little over a year ago, initially through friends and then through our use of Ray at Uber. More recently, I studied Anyscale and Ray in much greater detail and also had the chance to speak with many members of the Anyscale team. I see great potential in both the technology and the team. The company has a very realistic immediate vision and an ambitious long-term vision which completely resonates with me.

But another thing that excites me about my move to Anyscale is that I now have the opportunity to revisit a chapter of my career from my time at Google around 15 years ago. I had worked with MapReduce, which I found to be transformational for distributed data processing. But I felt there was a need to improve on this abstraction to better support distributed computing. One motivating example at that time was related to the management of keywords in the Adwords database. We had a large number of new keywords being introduced every day and obviously a much larger number of keywords already present. We needed to perform many operations each day on these keywords to ready the ad campaigns for serving. Keywords/campaigns in different states required different kinds of operations. This did not fit perfectly into the MapReduce paradigm where the same operations are performed on a large volume of data. Instead, a Ray-like abstraction of connected tasks and actors operating in different ways on different kinds of data would have worked much better.

While I did come up with a few related ideas at that time, I did not pursue them further because I decided to work on Search instead. Now I'm fired up to be part of a team that has taken similar ideas and productionized them. I look forward to helping make Ray super easy to use, making it robust and resilient, and optimizing its various pieces. And hopefully make Ray a household term in the software engineering community.

There are countless examples of Ray making distributed computing easier and faster in real production situations. You can find many of these in the Ray docs and on our website. Most of these examples are from the AI space, where Ray has demonstrated a very clear and significant value add. In the long run, I expect to see the use of Ray encompass many more distributed system use cases beyond AI — just like MapReduce was initially conceived as a way of building Search indices faster and more easily (e.g., easily supporting concepts such as PageRank and TFIDF), but then vastly expanded for use with all kinds of data processing workloads.

Our founders Ion, Philipp, and Robert are brilliant and humble people, and have created a strong team of similarly brilliant and humble people around them. We take our work very seriously, but we also know how to have fun. We have game nights, holiday parties, and fun offsites. Not to mention nice touches such as a daily boba run to complement our lunches and dinners.

In closing, there's nothing more exciting than bringing prior experiences to bear on something new. I’ve worked on many distributed systems projects in the past, as well as worked on multiple aspects of search engines. I see a lot of connections with the work at Anyscale. Who knows, maybe one day we will see Lucene running on top of Ray — after all search engines are distributed systems!

Interested in joining us on our mission to make distributed computing simple and accessible to every developer? We’re hiring!