Scala Days 2013 / Scala for Data Pipelines

Please RSVP for instructions on how to join the event.


Scala's mix of functional and object oriented programming paradigms is ideal for building data pipelines. Many smart people have noticed this, and as a result, we now have three distinct APIs for creating MapReduce pipelines in Scala (Scrunch, Scoobi, and Scalding), as well as a standalone data processing framework developed purely in Scala (Spark). Although all of these tools showcase the power and elegance of Scala, there is a larger opportunity for the community to give pipeline developers a single API to use for both in-memory and batch-style data processing. We'll discuss some of the challenges in creating an API that balances the features of both in-memory and batch processing and demonstrate the benefits that this creates in ETL development, exploratory data analysis, and machine learning.


No outline is available

Content is not yet available

Josh Wills (josh.wills)


No biography is available.

For a complete view of this profile, including education, work experience and developer information, you need to be logged in and have a subscription.

Josh's upcoming trainings

No events

Josh's past online trainings

Josh's blog posts

IntelliFactory Offices Copyright (c) 2011-2012 IntelliFactory. All rights reserved.
Home | Products | Consulting | Trainings | Blogs | Jobs | Contact Us | Terms of Use | Privacy Policy | Cookie Policy
Built with WebSharper