Answer Software Corporation Home Page – SynCognito – Enquire – HyBase - Company Info - Contact Us

The HyBase™ Object Management System

A Dynamic Persistent Object System with Integrated SQL

©1995 Answer Software Corporation

HyBase is a general purpose application development tool. Its primary features are that it has a dynamic, p-code interpreted object oriented programming language (OOPL), that it is an object database (ODBMS) with fully integrated SQL, that it has a client/server architecture, and that it is small and efficient.

HyBase can fill many different roles in support of an application. To some it is a Smalltalk-like dynamic language where all of the objects also happen to be persistent. To others it is an object database with a convenient dynamic query language. To others still, it is simply a convenient desktop SQL server, one with the added advantage of providing a simple, sensible glidepath for injecting object oriented features to an initially relationally oriented application.

In order to get a better sense of the role that HyBase plays in the development of specific applications, it is helpful to review the current state of the art for object orientation in general.

Object Orientation and HyBase

Object orientation (OO) is a difficult phenomenon to pin down. It has been bubbling for twenty years, a lifetime by software standards, but is even still a very broad and loosely defined discipline. This is a perfectly natural result of the core benefit provided by object orientation: its ability to support tremendous variation that approaches, at least in theory, the complexity of the real world itself. It should thus not be surprising that a tremendous variety of OO tools have also appeared sporting different methodologies, different styles, different emphases, different objectives.

Interchangeable Parts

For application developers, the heart of the OO revolution is something akin to the inspiration of Henry Ford's assembly line: the ability to build and use interchangeable parts. For software, the goal is to create chunks of software that can be built, tested, distributed and applied in a largely independent manner. The payoff for interchangeable software is very nearly the same as for interchangeable oil filters.

The design and construction of complex software applications, as with cars, can be broken down into largely autonomous smaller parts. The smaller pieces are easier to build and maintain because they have no, or at least greatly restricted, dependencies on the larger more complex picture. In OO-speak, this is called "encapsulation."

Once a part has achieved a reasonably clean and robust design, it can then easily be used to construct many other kinds of things. Those products in turn benefit from being built out of dependable and predictable elements. This dynamic applies as much to software as to O-rings and in OO-speak is referred to as "reuse."

A critical aspect of interchangeability is the ability to make a change dynamically. A static system, such as a compiled language like C++, benefits from encapsulation and reuse in the sense that its elements are modular, but it is not possible to change elements "on the fly" as with replacing one type of air filter or spark plug with something better suited to an individual unit on the production line. True interchangeability implies a fluid, dynamically changing behavior and therefore a dynamic language like HyBase.

Components

As with both cars and software, encapsulation implies that a much greater level of specialization is possible. This means that software developers of all sorts, whether in-house or third party, can focus on amassing expertise in selected areas rather than being stretched into knowing a little about a lot. At present this expertise is given form in a largely static way as code libraries, primarily for C++. In the not-too-distant future it will take the form of software "components", smallish processes that excel at performing a particular function and communicate with each other through something like CORBA, OLE or OpenDoc. Robust comprehensive applications, the software equivalents of Corvettes, can be assembled much more easily and safely from constituent components compared to the old way of building monolithic, unmaintainable monsters.

Many products currently exist that provide a single overarching "one-stop shopping" type of environment in which to build applications. On the surface this looks like a good idea by providing a simplified component-like approach where the application developer can benefit from the expertise embodied in the development environment. In reality this approach forces the developer into an uncomfortable choice: either a) choose a least common denominator product that performs all the required services in a mediocre way, or b) choose a product that is robust in some features while abandoning others.

Future applications will be less centralized, more holographic creations that involve the interaction of many different independently created software components. Each component will be used for that part of the application that it does best. This allows application development to move from a least common denominator to a best of breed mindset.

Even though HyBase can serve many powerful functions, it is not an application development tool in the currently popular "one-stop shopping" sense. For example, there are no GUI elements to HyBase beyond some simple configuration and monitoring tools. New independently developed GUIs seem to pop up every day and they are all truly as wonderful as they claim. Because of this incredible variation, both over product lines and over time, it would be a serious hazard for an application to be married to any single method of user presentation.

Three Tier Applications

The freedom created by the use of components allows applications to be partitioned in many different ways across their elements. One currently popular model is "client / server", where one component is largely responsible for user interaction and a second component is responsible for data storage.

The emerging "three tier" model provides a slightly more refined scheme by inserting in between the old "client" and "server" elements a third element intended to focus purely on abstract functions such as data derivation. Specifically the three tiers are : the presentation tier, the application tier and the database tier. Components in the presentation tier focus exclusively on information display and user interaction. Components in the application tier focus exclusively on data manipulation and derivation. Components in the database tier focus exclusively on data storage and retrieval.

HyBase fits very naturally into either the application tier or the database tier and could even be used to perform the functions of both, either for small application installations or for use during a prototyping phase.

The value of "three tier" is that it provides a discipline that aids in application management, much as SQL provides a useful discipline for database management. Both are techniques that are highly appropriate and useful in some cases, unnecessary and awkward in others. All object oriented applications are naturally multi-tiered creations where each component corresponds to a tier. The intention of HyBase is not to enforce or even prefer any one component or tier model but rather to be as supportive as possible within whatever model is sensible for a given application.

Language

Object orientation can be a double-edged sword. Encapsulation in practice requires the ability to create new datatypes together with their own special behaviors essentially at any time. This directly conflicts with the use of user-friendly graphical environments whose core assumption is that they can anticipate in reasonably complete detail the kinds of data queries and manipulations a user might want to perform. To the extent that some behaviors evolve into standards, for example ODBC-style database queries, these can and certainly will be incorporated into point-and-click environments. But a great deal of important functionality, such as company-specific business objects, must necessarily remain outside the reach of graphical development tools beyond using them simply as presentation media.

This means that OO development requires an unavoidable degree of programming. Some graphical programming tools have arisen hoping to meet this need. At present most are essentially fancy editors that still require a solid understanding of the intricacies of the underlying language. Some tentative attempts, still very much in their infancy, have been made at purely graphical programming tools with no implied underlying language. The current leading OO languages, C++ and Smalltalk, are a boon for heavy-duty developers and close to illegible for most everyone else. Virtually all of the otherwise "user-friendly" OO languages do not support the crucial ability to create your own data types with their own behavior, at best supporting user-defined record structures if that.

The HyBase language has been made as familiar and natural as possible without compromising functionality. Arithmetic expressions follow the normal precedence rules, unlike Smalltalk. Simple parametric stored procedures, called "tools" in HyBase, can be created as well as traditional object oriented methods. Particularly important from a database point of view, SQL is a fully integrated part of the HyBase language.

The dynamic nature of the HyBase language adds to this accessibility. It is easy to test out new or unfamiliar methods or tools on temporarily created test objects. Methods or tools can be dynamically changed, due to bug fixes or feature extensions, and then immediately retested and used. Adhoc queries constructed on-the-fly by other application components can be arbitrarily complex and detailed.

Persistence

The currently predominant technique for providing persistence to OO applications, usually C++ applications, is through the use of object stores. Their purpose is essentially to take an object identifier that is meaningful to the store, for instance a 64 bit number, and to do whatever is necessary to make the instance data associated with that object be loaded into local memory. This mapping of object ID to local memory may involve the use of data distribution, security checks and transactions.

This can be a very simple and powerful technique but unfortunately also results in a fundamental disconnect between data and its behavior. The data lives in the object store while the functionality lives in a separate C++ application image. This is not a tradeoff to be taken lightly because the very foundation of the object oriented model rests on the tight marriage of data and functionality within objects.

In HyBase the class definitions, methods and stored procedures are objects themselves that are stored right alongside the instances that they define and manipulate. This is made possible in part by the fact that the HyBase language is based on an interpreted p-code engine. The p-code definition of a method or tool may safely be stored in the database in a platform independent way.

There is no distinction in the HyBase language between persistent and non-persistent objects. Everything is transparently persistent. There is no need to consider load and store methodologies or timings since all objects are implicitly swapped in and out of memory on a last-touched basis. Consequently, application logic does not have to be consciously scaled to anticipated RAM configurations as one might have to with a C++ or Smalltalk application.

Although HyBase is a robust, full-featured ODBMS (Object DBMS) with multiple levels of transactions, it does not compete against the high-end ODBMS's in all features. In particular, HyBase does not provide automatic distribution of objects across systems. On the other hand, HyBase is a far more resource-friendly process that comfortably coexists on an average-sized desktop computer with other common software products. This is a claim that most non-persistent OO environments cannot match, much less the high-end ODBMS. In practice, HyBase is a very scaleable ODBMS that runs from the very smallest desktop computer up to the workstation level, whereas the other ODBMS's are scaleable starting only at the workstation level and going up.

Object-Relational "Hybrid" DBMSs

There is a budding industry of DBMS products that are called "object-relational" or "hybrid" databases. The mix between the object and relational aspects are invariably highly unequal and therefore of limited value. Either an object oriented front has been cobbled onto a fundamentally relational engine, or a relational front has been cobbled onto a fundamentally object engine.

HyBase is truly unique in the degree to which the object and relational worlds are merged. There are a number of factors that contribute to this. SQL is fundamentally an adhoc language, and HyBase supports adhoc (dynamic) as well as compiled (stored) queries. SQL requires the ability to execute, with reasonable performance, adhoc joins between tables where the specific join might not have been anticipated by the developer. The key to this is the implicit maintenance of indexes on various key fields, and HyBase supports implicitly maintained indexes and will appropriately optimize SQL commands to use them. Conversely, the standard ODBMS model presumes that tables are "joined" by constructing records that contain pointers that point directly to related objects. HyBase can do this also, and additionally allows the creation of user-defined pointer types which can for instance perform user-defined searches and validations.

The standard SQL syntax is a completely integrated subset of the HyBase language. The familiar-looking (to SQL users) SELECT commands may be used to access both traditional and user-defined datatypes. The functionality of the WHERE clause can be transparently extended by creating user-defined comparison operations on user-defined types, but it has also been extended to support any legal HyBase, ie. any possible object oriented, boolean expression. SQL commands may be applied to any collection-type object. Likewise, SQL commands may be used inside methods or stored procedures and, as we have just seen with the WHERE clause, also invoke methods or stored procedures.

HyBase also has a highly refined system for supporting implicit coersion between datatypes. This is an important glue for bridging the SQL and object worlds. SQL is fundamentally classless and constantly generates new data objects that essentially have implicitly defined classes, for example the results of SELECT statements. Classical object orientation, on the other hand, can be very unforgiving about even slight mismatches between classes. HyBase provides a natural means by which a developer can define coersion methods that may be implicitly applied when a situation demands that an instance be coersed from one class to another. Many useful coersions are built into HyBase, including coersion of any object to text, coersion between the many different native date and time types, and coersions based on structural similarities such as between similarly looking records.

HyBase Usage Scenarios

The following are sketches of different ways in which HyBase's unique combination of features can add particular value to applications.

Scriptor with User-defined Types

The simplest use of HyBase is strictly as a high-level scripting language for performing basic system and application management functions. The HyBase language already has the feel of a high-level scriptor, and it can be further extended with user-defined classes that represent high-level aspects and features of the system.

For example, with a TCP stream object in HyBase one could write a simple script, a HyBase tool, that sends the contents of a picture file directly to a client site.

Desktop SQL RDBMS

Despite its powerful OO features, HyBase is often used simply as a convenient, easy to configure desktop SQL server. The application's GUI is written to send strictly ANSI SQL commands to HyBase. This also allows HyBase to be used in tandem with another higher-end SQL server running on a mainframe or workstation. HyBase is deployed in the smaller environments where the high-end servers do not fit.

Desktop ODBMS

HyBase can function as a purely object oriented DBMS that is desktop resident. As such it could be used as a tool for gaining experience with the issues involved in building ODBMS-based applications and as a prototyping tool. HyBase is also a good choice for providing the ODBMS function for desktop-oriented vertical applications.

Data Access Integration

Special HyBase objects can be created that look and act like native HyBase tables but that in fact derive their records in an active, real-time way from other data sources such as mainframe SQL servers, legacy databases or even spreadsheets and live network feeds. Creation of such "foreign table proxy objects" in HyBase provides a single unified mechanism for accessing otherwise disparate data. Other application components then only need to support a single interface and a single paradigm in order to access any data relevant to the system. It is even possible using this mechanism to do multitable joins in HyBase between tables that reside in otherwise completely unrelated data sources.

Migration Path from SQL or Legacy Data to OO

Even if you are sold on the idea of "going object oriented", moving applications from SQL-based or legacy -based structures toward OO can be much more immersion than migration. Most OO tools and databases require a radical restructuring of preexisting application structures and paradigms into their own models. HyBase provides an opportunity to move into OO in a more measured, gradual way.

One might initially build a strictly traditional SQL database in HyBase and then gradually extend it in object oriented ways by creating and using special field types. This is discussed in some detail in the next section.

For existing applications, objects that act as active proxies for tables in foreign databases can be created. Existing data need not be immediately imported and restructured in order to benefit from and gain experience with applying objects and object methodologies to it. This is especially important for the data generated and accessed by live production systems. And because HyBase can operate on desktop computers, there is no need for a large up-front investment in big iron to experiment with an unfamiliar technology.

Three Tier Application Layer

HyBase functions very naturally as the application layer of a three tier application. Foreign table access objects provide an intuitive interface to database tier components. Text and other coersions provide a convenient means for managing display data generated for the presentation tier components. Full support for OOPL features such as multiple inheritance, class variables and a dynamic language provide real power for generating complex reports and responding to intricate queries.

Using HyBase Object Orientation to Extend SQL

One of the most constricting features of SQL is its limited array of datatypes available for describing the columns of tables. This section explores a range of techniques for extending traditional SQL with user-defined types in HyBase. These techniques may be used when creating new SQL applications in HyBase. They are also extremely useful in extending and redefining the behavior of data imported from other data sources.

Simple Record User-defined Type

It is not uncommon in SQL applications for a set of columns to appear in multiple tables. For instance, several tables may have columns named "Street", "City", "State" and "Zipcode". A single HyBase class named Address may be defined that has these four columns as its own elements. The four old columns may then be reduced to a single column having the new class. This change in table definitions is completely transparent to all SQL commands with only two exceptions: the CREATE TABLE command is changed to define a table with the new class, and the all-columns form of SELECT ("SELECT * …") will return the entire address object as a single column.

For example, these SQL table declarations:

Create Table Employee ( Name char(30),
	Street char(20), City char(15), State char(2), Zipcode char(9),
	… )

Create Table Office (
	Street char(20), City char(15), State char(2), Zipcode char(9),
	Department char(4),
	… )

could be declared after defining this HyBase class:

define class Address
	[ Street : [20]char, City : [15]char, State : [2]char, Zipcode : [9]char ]

to look like this:

Create Table Employee ( Name char(30), Address address, … )

Create Table Office ( Address address, Department char(4), … )

 

In addition to simplifying the declaration of the tables, this is a good way to guarantee that multi-field values are properly declared in multiple tables.

Custom Text Formatting and Input

Even for otherwise "normal" data that is stored in a single column in a common SQL type, it is often desirable to have better control over how data is displayed or parsed from text. The user interface can and frequently does fulfill this function, but it is often both safer and more convenient for the database to do it especially if more than one user interface is or may be involved in the application.

Special methods may be created that tell HyBase how to perform implicit coersions of objects from one class to another, and in particular to and from text. This in effect provides an application developer with complete programmatic control over these functions.

Consider a database with an integer field that represents the number of days since September 15, 1954. That odd sounding day was when the company was founded and all fiscal accounts have been clocked from that moment. It is also a convenient reference point for the application because September 15 is the start of this company's fiscal year. Without any special handling, this field would display as a number. For instance the date March 20, 1993 would display as 14073, the number of days since 9/15/54. Conversely, inputting a date value into this field would normally result in some sort of input error because the field after all is not a date but an integer.

With HyBase, you could do the following. First define a new class, we'll call it CompanyDate, that inherits from int. This means that the new class will be and do everything that the int class does. Then define methods on this new class for coersing to and from text. In this case it is both convenient and desirable to also define methods for coersing to and from date. Simple arithmetic can be performed on HyBase date objects and this makes it easy to calculate the real date from the stored integer. The text coersion methods are then defined based on the date coersions.

The methods named ":date" and ":string" are for coersing into dates and text, respectively. For these methods, "self" represents the target CompanyDate object. The methods named "date:" and "string:" are for coersing from dates and text. For these methods, "self" represents the target date or text object. The following is the complete HyBase code necessary to implement the CompanyDate class.

/* Define the new class as inheriting from the 'int' class */
define class CompanyDate [inherit int]

/* Create a class variable that holds the 'baseDate' constant */
define classVariable BaseDate on CompanyDate "September 15, 1954":date

/* Create the methods for coersing to and from dstamps */
define method ':date' on CompanyDate CompanyDate.baseDate+ self
define method 'date:' on CompanyDate self - CompanyDate.baseDate

/* Create the method for coersing to and from text */
define method ':string' on CompanyDate self:date:string
define method 'string:' on CompanyDate self:date:string

Now tables containing this type of date value may be created as:

Create Table LedgerEntries( ledgerDate CompanyDate, … )

All other SQL commands will work exactly as when the "Date" field was declared to be an int, except that the field will now display as a date.

An additional benefit of this approach is that comparisons can now be made between this field and other date fields or date literals. Imagine that this application also has another table with a normal date field in it, one that was created like this:

Create Table Checks ( checkDate date, … )

Without the HyBase coersions it would have been impossible, or at least very cumbersome, to relate the two tables on these columns. Now this can be done simply and directly like this:

SELECT … FROM LedgerEntries, Checks
	WHERE ledgerDate > checkDate
		AND …

Computed Fields

Methods can be defined on record classes that look and act just like fields in the record except that the value is actively derived by the method rather than extracted from a field in the record. Imagine an investment record as defined below. A method named "TotalPrice" can be created that computes its value by multiplying the price per share by the number of shares. It can then be used anywhere a normal field can be referenced as in the SELECT command below.

define class Investment [ Name : string, SharePrice : real, NumShares : int ]

define method '.TotalPrice' :real sharePrice * numShares

create Investments : table of Investment

SELECT Name, TotalPrice FROM Investments WHERE TotalPrice > 10000

As a more involved example, one could implement a "CurrentPrice" method that actually does a real-time look up of the current stock price from a Reuters feed.

Computed Tables

It is also possible in HyBase to create special classes that represent lists or tables whose elements or rows are computed on-the-fly. These elements themselves may largely consist of computed fields, possibly simple referents to rows and fields in other tables, possibly something much more complex. The SQL SELECT, UPDATE and DELETE commands operate on these tables just as well as on the more traditional structures. One reason for doing something like this is to provide a way for SQL-oriented presentation tools, specifically tools with built-in support for ODBC, to access otherwise non-relational, or at least non-SQL, data.

Case Study: Treasury Systems and Hybase at Apple Computer

[ The following case study was provided by Dan Farrand, the MIS manager for the Treasury group at Apple Computer. ]

 

The Treasury group at Apple is made up of 8 people in Cupertino and 3 in Paris, France. The Treasury group has primary responsibility for managing cash assets on the balance sheet and has a significant impact on the income statement in the other income and expense line. Specific responsibilities include cash management, investments, borrowings and management of FX (foreign exchange) exposures worldwide. In fulfillment of these responsibilities Apple Treasury is active in FX and money markets, each year entering into 4,000 - 6,000 FX, investment and borrowing contracts and initiating 7,000 - 10,000 cash movements.

Automated back office support for this activity is vital. Without solid, reliable back office support, Treasury operations would entail much more risk and the ability of Treasury to act would be severely limited. To address this problem, we have developed, in-house, a system that we call SAVANT. A "savant" is a person with particular expertise within a domain, in this case back office operations.

SAVANT is client/server based. Currently the main server is SyBase running on a Sun 3. Clients are, of course, Mac’s with client software implemented in HyperCard using an interface to SyBase called HyperDBLib. Hybase is used specifically for accounting/reporting on the FX, investment and borrowing portfolios. Hybase is particularly suited to this role because of its support of object-oriented concepts such as inheritance and polymorphism.

A portfolio is a collection of contracts. Contracts share many common attributes but differ in detail. The way you calculate or account for accrued interest on a time deposit is different than for a bond or a floating rate note. In a traditional environment, you inevitably find some variation of

if ContractType = "BOND" then CalcAccruedIntrForABond( )
else if ContractType = "TD" then CalcAccruedIntrForATD( )
else if ContractType = "FLOAT" then CalcAccruedIntrForAFloat( )
else …

This makes the process of reporting much more difficult. When new instrument types come up, you are faced with major modifications to the system. In an object-based environment, it’s simply a matter of subclassing our generic contract class and overriding standard behaviors in the new subclass. Now the process of accessing the accrued interest of a portfolio only involves:

select Portfolio AccruedInterestFor ( CurrentPeriod )
	from Portfolio
	where Portfolio ActiveThisPeriod ( CurrentPeriod )

With this select statement we visit each object in our cash portfolio table. In the where clause we ask each object in the portfolio if it is active in the current period. If an entry is selected by the where clause, we calculate and return the accrued interest for that object . ActiveThisPeriod and AccruedInterestFor are "messages" sent to each object in the portfolio and are calculated in the appropriate way by the object itself. CurrentPeriod is an object that specifies the current period.

The Portfolio table is filled with objects of different classes: BONDS, TD, CD etc.. Each of these classes is a subclass of CONTRACT. CONTRACT implements an AccruedInterestFor method that supports the general case of time deposits. Each of the other subclasses overrides the CONTRACT implementation with their own specific implementation.

In the above example you can begin to appreciate the expressive power of the object model. You can also see the integration of SQL and OOP (object-oriented programming) that Hybase supports.

Another important advantage of the object model is in support of adhoc query capabilities. The people managing our portfolios are always interested in looking at our cash in different ways. The information they are interested in changes over time. This information tends to be calculated and the calculations tend to require references to other tables such as coupon payment schedules, month-end FX accounting rates (to convert all investments to a dollar basis) or interest rate reset schedules (for variable rate instruments).

There are many front end tools that build SQL out the back-end, but they would all have a difficult time supporting adhoc queries to this kind of dynamic multi-relational data. With Hybase, we eliminate this problem because it treats method calls like virtual fields. With methods to implement dynamic calculations we can make the database appear to be virtually flat. Building adhoc query tools becomes relatively simple.

A final feature of Hybase that serves us well is its ability to interface to other environments. Answer Software provided us with an interface to SyBase. In this way Hybase can transparently synchronize its information with the master data in SyBase. Our users generally don’t know which database they are working with. HyperCard communicates with both SyBase and Hybase on the network and provides a common interface to the user.

We are currently planning to port the entire Treasury system to Hybase. We think that, given the relatively low transaction and data volumes, we can live without the large scale capabilities of SyBase and avoid the substantial expense of maintaining it. With Hybase we should be able to get reasonable performance on a Quadra 950 in an environment that is significantly more expressive and able to model the domain we work in more directly.

 

Dan Farrand
Treasury Information Manager
Apple Computer
(408) 974-6639
email: DanF@applelink.apple.com

Follow-up

Since the above report was written, the Apple Treasury group has indeed ported the remainder of the Treasury system to HyBase and is reporting the sorts of benefits that were anticipated.

System Requirements

Operating system: Mac OS 7.x, 8.x; Microsoft Win95

RAM: 2 megabytes

Disk: 4 megabytes

Home Page – SynCognito – Enquire – HyBase - Company Info - Contact Us Copyright ©2000 Answer Software Corporation