Yahoo Pipes

Yahoo! Pipes brings mashups to the masses

Tony Patton, TechRepublic

The Yahoo! Pipes project facilitates mashup creation via an easy-to-use interface. My first introduction to the concept was with the UNIX platform. UNIX Pipes allows you to route output from one process to another process as its input; that is, you can easily route data through a series of loosely connected applications. This simple definition is the core concept behind the Yahoo! Pipes project.

What is Pipes?

Pipes is a free online service that lets you remix popular feed types and create data mashups using a visual editor. The visual editor is a key component as it, theoretically, makes the service accessible to non-programmers. You may use Pipes within your Web applications, or you may publish and share your mashups as Web services without ever having to write a line of code.

In practice, this means that you can use Pipes to pick a few feeds or APIs to retrieve data from, set up rules for processing the data that is provided by those feeds, prompt for user input in your processing, and then output the processed results as another feed or object format for use in your own applications. In the simplest case, you can apply some straightforward rules to a feed and then subscribe to the end result.

The editor

One of the best features of Yahoo! Pipes is the user interface. It provides a drag-and-drop user interface to create your own Pipes. Even if you don't like the tool, you will be impressed with the editor. It allows you to create your own Pipes by using predefined items called modules on a toolbar on the left portion of the screen. Modules are the tools that enable the mixing, sorting, and merging of different data sources. The following list provides a sampling of the available options for building your own Pipes:

• Sources: Data inputs are set up using Sources. You can pull in RSS or Atom feed that may be accessed via HTTP.
• User inputs: You can specify input for a Pipe. The values may be numbers, URL, location, text, or date.
• Operators: There are a variety of pre-defined operators that may be used to process data. This includes loops, sorting, union, and so forth.
• Url: This module facilitates the creation of custom URLs that pass data to other modules.
• String: This module provides various string operations such as regular expressions, translation, replace, and concatenation.
• Date: This module provides functionality for formatting and building dates.
• Number: This module provides simple math functions like addition and subtraction that can be used to manipulate data.

You may drag and drop these modules to the screen's layout area. Lines are used to connect modules together (i.e., using the output of one module as the input for another). The editor includes buttons that allow you to work with Pipes, such as create a new one, save your work, create a copy of the currently open Pipe, and view/edit properties of a Pipe.

The screen's bottom portion is a debugger area that provides feedback. The debugger allows you to step through your modules one at a time and view the data at each step -- allowing you to easily track down any issues or problems that may occur. The editor uses a tabbed interface, so you can work on multiple Pipes at one time -- each opened within its own tab.

A simple example of a Pipe is using an RSS feed as the data source (Source module). Each data item in the RSS feed can be processed in a loop (Operators module). Each data item in the loop may be used as an input for a Yahoo! search (Source module). The output of the Yahoo! search using data items from the RSS feed is the output of the Pipe. The Pipe output may be consumed as RSS or JavaScript Object Notation (JSON). In fact, you may publish the Pipe for others to use. This is a simple example, but it does provide a peek at its functionality.

Considerations

While the tool is described as easy to use, it still requires a bit of technical knowledge to properly use. For instance, you need to be familiar with URLs and their various parts if you want to take advantage of the User inputs modules; it also requires some expertise to work with the various data types and other features. While you don't have to be a hard-core developer, you must have a good understanding of the Web.

A big issue with this type of tool is browser support, especially given its impressive interface. Basically, there are some compatibility issues with older versions of some browsers. With this tool, it helps to have the latest version of any browser. I have used it with Internet Explorer 7 and Firefox 1.5 and 2 with no problems.

Centralise data

Yahoo! Pipes is simple and accessible to use. The visual editor makes it easy to jump in without any coding. It embraces Web 2.0 concepts as it facilitates the creation of new Web services or feeds using existing components or sites. It allows you to combine them in novel ways to provide whatever is needed.