Cascading (software)
Software abstraction layer for Apache Hadoop and Apache Flink
title: "Cascading (software)" type: doc version: 1 created: 2026-02-28 author: "Wikipedia contributors" status: active scope: public tags: ["free-software-programmed-in-java", "free-system-software", "cloud-infrastructure", "apache-hadoop", "software-using-the-apache-license"] description: "Software abstraction layer for Apache Hadoop and Apache Flink" topic_path: "general/free-software-programmed-in-java" source: "https://en.wikipedia.org/wiki/Cascading_(software)" license: "CC BY-SA 4.0" wikipedia_page_id: 0 wikipedia_revision_id: 0
::summary Software abstraction layer for Apache Hadoop and Apache Flink ::
::data[format=table title="Infobox software"]
| Field | Value |
|---|---|
| name | Cascading |
| latest release version | 3.3.0 |
| latest release date | {{cite web |
| url | https://github.com/Cascading/cascading/releases |
| title | Releases · Cascading/cascading |
| website | github.com |
| access-date | 2021-03-29 |
| latest preview version | 4.0-wip-120 |
| latest preview date | {{cite web |
| url | https://github.com/cwensel/cascading/releases |
| title | Releases · cwensel/cascading |
| website | github.com |
| access-date | 2021-03-29 |
| programming language | Java |
| license | Apache License v2{{cite web |
| url | https://github.com/Cascading/cascading/blob/3.3/LICENSE.txt |
| title | cascading/LICENSE.txt at 3.3 · Cascading/cascading |
| website | github.com |
| access-date | 2021-03-29 |
| repo | |
| website | |
| :: |
| name = Cascading | latest release version = 3.3.0 | latest release date = {{cite web | url = https://github.com/Cascading/cascading/releases | title = Releases · Cascading/cascading | website = github.com | access-date = 2021-03-29 | latest preview version = 4.0-wip-120 | latest preview date = {{cite web | url = https://github.com/cwensel/cascading/releases | title = Releases · cwensel/cascading | website = github.com | access-date = 2021-03-29 | programming language = Java | license = Apache License v2{{cite web | url = https://github.com/Cascading/cascading/blob/3.3/LICENSE.txt | title = cascading/LICENSE.txt at 3.3 · Cascading/cascading | website = github.com | access-date = 2021-03-29 | repo = | website = Cascading is a software abstraction layer for Apache Hadoop and Apache Flink. Cascading is used to create and execute complex data processing workflows on a Hadoop cluster using any JVM-based language (Java, JRuby, Clojure, etc.), hiding the underlying complexity of MapReduce jobs. It is open source and available under the Apache License. Commercial support is available from Driven, Inc.
Cascading was originally authored by Chris Wensel, who later founded Concurrent, Inc, which has been re-branded as Driven. Cascading is being actively developed by the community and a number of add-on modules are available.
Architecture
To use Cascading, Apache Hadoop must also be installed, and the Hadoop job .jar must contain the Cascading .jars. Cascading consists of a data processing API, integration API, process planner and process scheduler.
Cascading leverages the scalability of Hadoop but abstracts standard data processing operations away from underlying map and reduce tasks. Developers use Cascading to create a .jar file that describes the required processes. It follows a ‘source-pipe-sink’ paradigm, where data is captured from sources, follows reusable ‘pipes’ that perform data analysis processes, where the results are stored in output files or ‘sinks’. Pipes are created independent from the data they will process. Once tied to data sources and sinks, it is called a ‘flow’. These flows can be grouped into a ‘cascade’, and the process scheduler will ensure a given flow does not execute until all its dependencies are satisfied. Pipes and flows can be reused and reordered to support different business needs.
Developers write the code in a JVM-based language and do not need to learn MapReduce. The resulting program can be regression tested and integrated with external applications like any other Java application.
Cascading is most often used for ad targeting, log file analysis, bioinformatics, machine learning, predictive analytics, web content mining, and extract, transform and load (ETL) applications.
Uses of Cascading
Cascading was cited as one of the top five most powerful Hadoop projects by SD Times in 2011,{{cite news | last = Handy | first = Alex | date = 1 June 2011 | title = The top five most powerful Hadoop projects | url =http://www.sdtimes.com/content/article.aspx?ArticleID=35596&page=1 | newspaper = SD Times | location = | publisher = | accessdate = 26 October 2013 | last = Taylor | first = Ronald | date = 21 December 2010 | title = An overview of the Hadoop/MapReduce/HBase framework and its current applications in bioinformatics | url = http://www.biomedcentral.com/1471-2105/11/S12/S1 | newspaper = BioMed Central | location = | publisher = Springer Science+Business Media | accessdate = 26 October 2013
- MultiTool on Amazon Web Services was developed using Cascading.
- LogAnalyzer for Amazon CloudFront was developed using Cascading.
- BackType - social analytics platform
- Etsy - marketplace
- FlightCaster - predicting flight delays
- Ion Flux - analyzing DNA sequence data
- RapLeaf - personalization and recommendation systems
- Razorfish - digital advertising
Domain-Specific Languages Built on Cascading
- PyCascading - by Twitter, available on GitHub
- Cascading.jruby - developed by Gregoire Marabout, available on GitHub
- Cascalog - authored by Nathan Marz, available on GitHub
- Scalding - A Scala API for Cascading. Makes it easier to transition Cascading/Scalding code to Spark. By Twitter, available on GitHub
References
References
- "Cascading and Driven | Support".
- "Integrate.io - One Platform To Support Your Entire Data Journey".
- "Cascading modules".
- [http://codeascraft.etsy.com/2010/02/24/analyzing-etsys-data-with-hadoop-and-cascading/ Blog post by Etsy describing their use of Cascading with Hadoop]
- "Cascading User Guide".
- "Hadoop Application Performance Management - DRIVEN's Features".
- White, Tom. (September 24, 2010). "Hadoop: The Definitive Guide". "O'Reilly Media, Inc.".
- "Getting Started on Hadoop".
- "Julio Guijarro, Steve Loughran and Paolo Castagna, "Hadoop and beyond," HP Labs, Bristol UK, 2008.".
- "Flightcaster Presentation Hadoop".
- "NoSQL, Hadoop, Cascading June 2010".
- (2014-05-07). "Using Cascading to Build Data-centric Applications on Spark".
- "Cascading{{Not a typo".
- "AWS Articles".
- [http://tech.backtype.com/ BackType blog] {{webarchive. link. (August 25, 2011)
- "FlightCaster".
- "Ion Flux".
- [http://blog.rapleaf.com/dev/2008/09/05/goodbye-mapreduce-hello-cascading/ RapLeaf Blog] {{webarchive. link. (February 1, 2011)
- "Razorfish Case Study".
- (17 September 2021). "PyCascading is no longer maintained".
- (August 8, 2018). "Cascading.JRuby".
- (June 23, 2023). "Cascalog".
- (June 22, 2023). "Scalding".
::callout[type=info title="Wikipedia Source"] This article was imported from Wikipedia and is available under the Creative Commons Attribution-ShareAlike 4.0 License. Content has been adapted to SurfDoc format. Original contributors can be found on the article history page. ::