Feature

The need for local search in embedded consumer devices

As the available memory in portable embedded devices grows, designers are investigating low impact data management techniques to tag, store, locate, and retrieve local information.

By Dale Dasker, Hitachi Embedded Business Group -- EDN, 11/16/2007

Robust search capabilities have made the Web a powerful resource. Google, Yahoo and others have conclusively demonstrated the value of Web search, and are starting to make inroads into desktop search. But embedded devices need search, too. Today's mobile phones, MP3 players and PDAs contain a surprisingly large volume of data and content for their increasingly small sizes, and users are demanding the ability to access and manipulate their data and content in new and interconnected ways.

The challenge here for developers is that search and data management are not easy to implement—especially when the data being managed is beyond simple alphanumeric data, such as GPS coordinates, or when data and content is meant to be accessed by more than one application. When designing the consumer devices of tomorrow, developers need to take into consideration how best to accommodate not only the growing volume of data and content, but also accommodate consumers' craving to use their devices to enhance their daily lives.

Not a device, but an "experience"

Think of today's typical mobile phone—not only does it enable consumers to communicate anywhere, anytime, but most also include the capacity to store contact information, take and store photos and videos, perhaps even play music or provide GPS navigation services. As a whole, it's an impressive device that has changed the way we live. But making things "better" is what drives developers. What if, rather than each function or application having its own, isolated data store, they all shared data and connected content among themselves in ways never before imaginable?

The savvy developers and service providers that figure out how to do this will find themselves able to offer an enhanced, personalized consumer experience like no other. The ability for consumers to relate their various types of content (music files, photos, videos, contact information, etc.) along with personal and preference data, such as GPS location data, personal ratings on anything from a song to a restaurant to the characteristics of friends in a contact list, could end up profoundly changing mobile phones, PDAs, MP3 players and similar gadgets from simple "devices" to robust "experiences."

By incorporating data management at the core device level, developers can provide the ability for a device that could, for example, take a photograph, time stamp it, connect it to the person's contact information whose birthday was being celebrated, tag it with the GPS coordinates of the location, relate it to the song playing at the time and a personal rating of the restaurant in which it was taken. This ability also offers up potential new revenue streams for service providers that can develop innovative subscription services that enhance the consumer's end-to-end experience.

Evaluating third-party middleware

Now that we've established the potential benefits of incorporating robust data management and search functionality at the device level, where do developers start? The perceived benefits of an in-house RDBMS are often misleading. Rather than spending development cycles and budget on developing the "perfect" data manager for their application, developers today have a wide array of middleware available from which to choose. Many of these can be customized to suit a specific application or device. Employing a commercial-off-the-shelf (COTS) RDBMS saves time and money, and gets products to market more efficiently.

Whether an individual RDBMS is suitable for a specific project depends on a variety of issues. Here are some of the things to consider when evaluating a middleware product:

• Proprietary vs. Open Standards: Selecting an embedded RDBMS based on open industry standards is a key point. Finding an embedded RDBMS that supports standards such as SQL, C, C++ and the JDBC interface will make it easier for developers with any RDBMS experience to work with. Also, standards-based middleware keeps developer options open to future technology advances, rather than being locked into a proprietary platform. Also important is that it supports a wide array of real-time operating systems (RTOS) and remains current in its support of the latest chipsets.
• Space Requirements: Even though today's chipsets are becoming smaller and more powerful, RAM requirements remain a key concern for embedded device developers. Some embedded RDBMS can run in a few hundred kilobytes. Depending on variables such as the chipset used, Hitachi's Entier embedded RDBMS takes up just half a megabyte, with no increase at runtime and no adverse affect on performance.
• Performance: Some embedded RDBMS are designed with optimal performance in mind, offering anywhere from 50- to 1,000 times faster results than their competition.
• Special Functions: The more types of specialized search functionality (conceptual search; spatial, location-aware search; complex text search; alias handling, etc.) enabled by the embedded RDBMS, the more options developers have to develop unique services to differentiate their device. Additionally, developers evaluating their embedded RDBMS options should look for the ability for multiple applications to query the database simultaneously as well as the ability to accommodate rapid incremental updates.
• Development Time: When more functionality is incorporated into the RDBMS, developers have fewer lines of original code to write. The fewer lines of code that need to be originated leads to less original code to maintain, and a lower risk of introducing bugs that will need to be resolved.

Designing a cross-application schema

Read more In-Depth Technical Features

Once the decision is made to incorporate an embedded RDBMS into the device, the next step is to create the schema. The schema defines tables, fields within the tables and the relationships between them. A well-designed schema creates opportunities for the developer to create new and unique user experiences. Consider the mobile phone example. Rather than each individual application storing its own data in its own data store, placing all of the data into one data store with multiple tables makes it possible for each application to access richer data. For example, the contact management application now has access to calendar and photo data. The implementation of this can be accomplished by creating a schema with three tables (CONTACTS, CALENDAR and PHOTOS). The table structure would look like:

CONTACTS

FIRSTNAME	LASTNAME	ADDRESS	...	PHONENUM	...	BIRTHDAY	...
John	Smith	123 Main St		408-972-1234	...	6/10/2004	...
Miles	Smith	545 1st St	...	415-918-2323	...	8/03/2001	...
...	...	...	...	...	...	...	...

CALENDAR

EVENT	DATE	TIME	DURATION	ATTENDEES
Birthday Party	6/10/2007	1400	2:00	John Smith, Jane Smith
Dinner	6/12/2007	1900	3:00	Miles Smith, David Jones
Coffee	6/18/2007	0730	1:00	Jane Smith, Miles Smith

PHOTOS

PHOTONAME	DATE	TIME	LOCATION
IMG_01.jpg	6/10/2007	1410	45.5208,122.6742
IMG_02.jpg	6/10/2007	1411	45.5208,122.6742
IMG_03.jpg	6/10/2007	1420	45.5208,122.6742
IMG_04.jpg	6/18/2007	0745	45.5212,122.6754
IMG_05.jpg	6/18/2007	0746	45.5212,122.6754

With this information stored within a single RDBMS, a new application can be created that finds all photos taken at a certain calendar event. This would be accomplished with a simple SQL statement:

SELECT PHOTOS.PHOTONAME
FROM PHOTOS
WHERE CALENDAR.DATE = PHOTOS.DATE

The C-language application code that would execute this search is:

Allocate a connection context and a connection handle:
rtnc = e_rdb_SQLAllocConnect(&cnctContext)

Establish a connection to the RDBMS:
rtnc = e_rdb_SQLConnect(cnctContext, dbpath, USER, PASSWORD, &cnctOpt);

Allocate a statement handle:
rtnc = e_rdb_SQLAllocStmt(cnctContext, &hStmt);

Preprocess the SELECT statement:
rtnc = e_rdb_SQLPrepare(cnctContext, hStmt,"SELECT PHOTOS.PHOTONAME FROM
PHOTOS WHERE CALENDAR.DATE = PHOTOS.DATE");

Execute the SELECT statement:
rtnc = e_rdb_SQLExecute(cnctContext, hStmt);

Bind a Variable to the return set:
rtnc = e_rdb_SQLBindCol(cnctContext,       /* Connection context */
                     hStmt,                                      /* Statement handle */
                     1,                                             /* Column number */
                     TargetValue,                            /* Variable address */
                     BufferLength,                           /* Variable length */
                     Ind));                                       /* Indicator address */

Fetch rows from the opened cursor:
rtnc = e_rdb_SQLFetch(cnctContext, hStmt);

/* Repeat for all data. */
while ( rtnc == e_rdb_RC_SQL_SUCCESS ){
        photo = TargetValue;
        printf("%s ", &photo);
        /* Fetch the next row. */
        rtnc = e_rdb_SQLFetch(cnctContext, hStmt);
}

A foundation for tomorrow's devices

When done right, employing an embedded RDBMS at the core device level can provide consumers experiences that are richer than anything they've experienced before. The ability of third-party middleware based on open standards makes it easier for developers to integrate this technology in their next-generation devices, setting the stage for ever more innovative services that enrich consumer's lives.

By using a COTS RDBMS, developers can realize huge gains in time-to-market, due in large part to a reduction in the amount of code needing to be written, which also leads to a lower number of bugs detected and needing to be remedied. All this gets the device out the door and earning revenue more quickly.

Author Information

Dale Dasker is a senior field application engineer at Hitachi Embedded Business Group where he serves as a resource for customers looking to incorporate the Hitachi Entier embedded relational database management system into their devices. He graduated from California State University, Northridge with a bachelor's degree in electrical engineering.