Database Consulting Group LLC Blog

Progress on the Dataphor front has been slow but steady for a long time now. Recently, however, thanks to support from the community as well as targeted investment from Dataphor users, we have been able to make some significant progress. In addition to several defect repairs, many new features have been added.

Instancing

A running Dataphor server has always been called an instance, but recent changes have made configuring and administering multiple instances dramatically easier. Multiple instances can be installed and managed on the same machine without the need to run command-line installations, or manage configurations using config files. A new listener service has been added to make instance discovery possible from the client, and port configuration can now be managed completely server-side. In addition, several changes have been made to consolidate the data used by an instance and enable multiple instances to use the same executable and library directories.

Native CLI

A new, lightweight, potentially stateless CLI has been built to enable a Dataphor server to be accessed from applications without having to incur the overhead required by the standard CLI. Because it is intended as an alternative, not a replacement, the new CLI is not as feature-rich, but it eliminates the dependencies on the Dataphor code base and can be used virtually stand-alone. This will enable the Dataphor server to be reached from a much broader range of clients, such as mobile applications, Silverlight clients, and other environments where resources are at a premium. In addition, the stateless potential of the new Native CLI enables applications to make scaling decisions that were previously infeasible with the standard CLI.

Cross-Instance Query

The Dataphor server now supports the configuration of Server Links that can be used to query other running instances of Dataphor servers. The Dataphor Server coordinates transactions between the current instance and any number of connected instances to provide seamless nested distributed transaction support.

SQLite Catalog Support

The catalog persistence layer was abstracted to provide the ability to plug-in different catalog store implementations, and a SQLite catalog store implementation was built. The Dataphor Server can now be configured to use SQLite, rather than SQL Server Compact Edition to provide catalog persistence.

Mono Support

In order to enable the Dataphor server to be used on a Mono platform, we have taken steps to remove any platform-specific dependencies in the code base, as well as the client tools.

All these features are currently implemented in the main development branch of the Dataphor project. To try them out, simply get latest from the open source svn repository and build the Dataphor solution. We have consolidated the projects so that it's easier than ever to build and get up and running quickly.

For more complete discussion and up-to-date documentation about the latest features available in Dataphor, see the What's New in 2.2 page of the dataphor wiki.

Posted by Bryn at 03:03 PM | Permalink | Comments (0) | TrackBacks (0)

There are several features of WPF that are not present in Silverlight, some of which are pretty difficult to work-around or live without.  One such feature is the SharedSizeGroup property that WPF has for column and row definitions of a layout grid.  This feature made many advanced layout scenarios entirely declarative.  Consider, for instance, the situation where an outer grid defines a set of control groupings for a form.  Within each of the out grid's cells are text boxes with left aligned labels.

In this case, the inner grids can use a SharedSizeGroup property on the column containing the labels and the text will be aligned automagically.

We wanted to be able to accomplish basically the same thing in Silverlight, and maybe even see if we could go about making it a little more general.  In particular we wanted to divorce the functionality from the Grid so that it could be used in non-grid situations as well.

What we came up with was a combination of two classes: SharedSizePanel and SharedSizeGroup.  SharedSizePanel is a simple, single element container which has the optional properties WidthGroup and HeightGroup.  All panels associated with a particular group will be given the size of the panel with the greatest content size in that dimension.

<local:SharedSizePanel WidthGroup="{StaticResource Horizontal}" >
  <Border Background="Coral">
    <local:ResizeBox Width="30" Height="30" />
  </Border>
</local:SharedSizePanel>

A SharedSizeGroup is a simple class that can easily be instantiated in the resources section:

<UserControl.Resources>
  <local:SharedSizeGroup x:Key="Horizontal" />
</UserControl.Resources>

We wanted to allow widths and heights to be synchronized to the same value, which would allow such fanciful things as the width and height of a particular control to be synced.  That capability would add quite a bit of complexity to what is presently a very simple implementation so we held off.  

There were a few challenges that were encountered in building this control, including:

• Custom Measure and Arrange logic doesn't seem to work as expected as a descendant of ContentControl.  It worked as Panel descendant, but there doesn't seem to be a way to limit the panel to a single child, so fortunately it was found to work as a ContentPresenter.
• The group cannot and must not force measurement of the panels, so it must store what the measurement was when it happened.  This is true for several reasons including the fact that the "available size" is only known when passed as an argument to the measure override.
• As a content control, the VisualTreeHelper class must be used to get the actual element that the Content becomes.  Content of type object and you can't Measure an object.

In the end, the solution seems relatively simple and elegant, so fortunately I consider that all of the above worked out favorably. 

I haven't seen any indications that SharedSizeGroup will be introduced in Silverlight 3, and I'm not ready yet to dispose of my Silverlight 2 development environment to find out.  Anyone have version 3 installed that wouldn't mind verifying?

Download the code for these classes.  Download the WPF SharedSizeGroup sample.

Posted by Nathan at 03:29 PM | Permalink | Comments (0) | TrackBacks (0)

• DrawingVisual Brushes – this doesn’t seem like a big deal, but without this, doing something as simple as say, tiling an image, becomes very difficult.
• Relative source binding – binding from one property of one entity to a property of another has to be constructed in code.  Hard to believe that this didn’t make the list, but fortunately is in Silverlight 3.
• TemplateBinding to anything other than a root dependency property.
• The DataContext is not inferred from parent to child.
• SharedSizeGroup – this subtle, but oh so useful property of the GridColumn and GridRow Grid classes.
• Popup.PlacementTarget – Popups have to be absolutely positioned, which is unfortunate given that there are no dialogs or forms, so Popups can be a pretty important element.
• LayoutTransform – RenderTransforms are useful, but certainly don’t provide for what LayoutTransforms did.
• Left mouse click – What were they thinking?!  They could have had an instant advantage over flash.
• OverrideMetadata – This important aspect of dependency properties can be a pain to live without.
• AddOwner – Another pretty useful dependency property trick.
• Read-only dependency properties – Another property thing that’s been a pain to work-around.
• Property change notification – This one is a real bear.  There seems to be no programmatic way to know that a DependencyObject’s property has changed (ouch).
• No custom routed events – This one has been a real pain for me, but might not come up for most apps.
• GetFlattenedPathGeometry and GetPointAtFractionLength – path functions that are possible to work around, but a pain.
• Baseline in FontFamily
• OnRender – They seem to have some good arguments for this one, however.
• Keyboard Key to character code translation - that is, knowing that say Key.D1 translates to character '1'.  Once has to also check the Shift state of the keyboard, and basic keys like + are only in the PlatrofmKeyCode, which requires a series of hard-coded assumptions to deal with.

I also had a couple Silverlight 2 gotchas noted:

• If you rename a Silverlight project, watch out that the "Startup Object" property of the project is corrected too, or the app will fail to start with no explanation.
• If you have a web application to go with your Silverlight project, don’t add the Silverlight application as a reference, associated it under the Silverlight tab in the web apps configuration dialog.
• If you will be creating more than one instance of a UserControl, don’t name it or you will receive and error when instancing the 2^nd.  This is a royal pain because if it is unnamed it cannot be referenced by the animation objects (or by-name binding in Silverlight 3).  Basically you have to programmatically create the binding for anything that binds to the root UserControl.

Links to other similar posts:

• List from Justin Chase on Silverlight forums.
• Work-arounds for missing Silverlight features on StackOverflow.
• Emulating CommandSource by Josh Smith
• Official list on MSDN

Posted by Nathan at 10:20 PM | Permalink | Comments (4) | TrackBacks (0)

DCG has been very quiet for some time.  Not much activity on any of our sites, and our dataphor.org and alphora.com sites have been down for a couple weeks.  Though this doesn't look good, I assure you we're still around and nutty as ever (speaking at least for myself).

I'll take this opportunity, being that this is the first blog post of the year, to give an update of where we stand and where we are going:

• Dataphor - Yes we still love Dataphor and continue to support and incrementally improve it.  The dataphor.org and alphora.com websites are down because we basically have one server and it was in need of a software rebuild.  You can thank all of the nice people who have nothing better to do than troll for vulnerable wikis and web servers.  You see, we DO have something better to do than patch and counteract said nice people, which is why our server reached the point of crippled.  Anyway, we are trying to do a little better job this time to lock everything down properly and reduce our "surface area". 
  With some community Tender Loving Care (TCL), Dataphor really could be a big success, but quite frankly with only us it would probably never break out of it's little niche.  That said, it's really not a bad niche to be in, and at the end of the day it gives us an advantage regardless of it's popularity, or lack thereof.
• Consulting - Consulting is presently our bread and butter as a company, and we count ourselves lucky to have some nice clients to work with and interesting projects to work on.  In the financial service world we're working on a hardware interface, business intelligence, lending systems, and other such things.  We're wrapping up a medical records system and starting a medical scheduling system.  We have also been doing some other smaller consulting projects.
• Mystery Project - Shh, part of our quietness is related to a secret project.  We should be able to make some kind of announcement on it in as early as 2-3 months when we hope to go beta.  No this is not some kind of Dataphor replacement.  This is an entirely different product for an entirely different purpose.  Fine, I'll say one little thing: it is relational, at least in spirit.  We are very excited about this project, and hope that we don't take the good idea and botch it up too badly in execution.

Long term, we still have our sights on big things.  We would like to do something about the complete insanity we call modern software; or at least provide a vehicle for those that think the same way.  Once you start down the path of realizing that there are fundamentally better ways to do things, you'll never be content with the status quo; thus is our lot.  We plan to give options for those seeking more abstract, powerful, and mathematical solutions to data management (hint, all software problems are data management problems).  It won't be for everyone, but the rest of the industry can continue to swim in a vat of their own complexity if they so choose.

Here's to 2009, the best year yet!

Posted by Nathan at 10:15 PM | Permalink | Comments (2) | TrackBacks (0)

ftp://ftp.research.microsoft.com/pub/tr/TR-2008-69.pdf

This excellent paper from James Larus at Microsoft Research discusses how software abstractions and raised software expectations have essentially squandered the hardware advances of the last 30 years.  Okay, perhaps squandered isn't the term, but the point is that software doesn't run any faster than it used to.  James points out legitimate reasons as well as less legitimate reasons for this, and looks to the implications of increased parallelism on all of this.

My take: On one hand, I am all for meaningful software abstractions.  I am also for language features that prevent us from shooting ourselves in the foot in obvious ways.  I think we could do much better, however:

• Missed abstractions, particularly relations, has led to excessively complex and bloated software. 
• Primitive data model - Today's software runs in a model based on allocating memory chunks and procedural APIs.  This leads to countless redundancies in the software stack, as well as poor interoperability and optimizability.
• Physical data dependence - by separating the logic from the implementation, we could continually improve software performance without rearchitecting the logic.
• Memory utilization - Garbage collection introduces more problems than it solves.  Value-based semantics and the relational model eliminated the need for GC.  Virtual memory is, in hind-site, a bad idea.  A much more effective architecture is to utilize memory as a cache for what are conceptually persistent data structures (e.g. DBMS architecture).
• Concurrency - again, if the programming model were shifted from references to memory to shared data structures, the issues with thread, process, and network concurrence are already solved.  Much more highly concurrent application are possible in ways that are either transparent or opaque to the developer.

If you've ever sat there for a minute or two waiting for an application to swap back into memory so that it can close, you probably know something is fundamentally wrong with today's architecture.

Posted by Nathan at 10:58 AM | Permalink | Comments (0) | TrackBacks (0)

It may seem that legacy--that is "old style" systems we've inherited or maybe even built but no longer like--is purely something we must live with now and forever.  It seems inevitable that our current systems, languages, and applications are doomed to eventually collect dust and an occasional deriding comment from some new intern who doesn't even know what it is like to boot a computer using floppies.  I say, however, nay!  How can I say this considering that it's always been this way!?  Stay with me.  First let's establish that some things never go out of style.  Like math for instance.  Math hasn't changed much since computers were born, and isn't likely to any time soon... or late.  Philosophers are still trying to decide whether math is somehow part of the universe, or if math is merely a cool abstraction we use to predict things about the universe.  Regardless, it certainly does give us a useful tool for modelling and predicting actual things.  Because of this, it has a timeless nature to it, even though we might change what symbols and tools we use to represent things. 

Computer software abstractions are unavoidably related to math, though in many cases the relation may seem like a cousin-in-law's friend's neighbor once removed.  It's actually this distance from more foundational principles that makes software so easily dated.  In the name of problem solving us Software mechanics have evolved some awfully patched up monsters.  This has been made possible by the kind hardware folks, who have been generous enough to make our obtuse layers of bandage-work actually perform in a reasonable manner (thanks guys).  One of the many prices to pay for so much bad software abstraction is the rapid manner in which our systems become legacy.  Legacy because they are so difficult and fragile to change that nobody wants to touch them.  Legacy because they are so laden with complexity that it is nearly impossible to tease out and salvage the essence as platforms change.  Legacy because we've got a newer better technology wheel we've reinvented and we must rebuild the old systems because now they just look old.

It's all of the non-essence stuff in our systems that makes us have to replace rather than extend or enhance them.  Think of any given system as being made up of two parts:  One part describes the essence of the problem for which the system was created, something like mortgage amortization, modeling customers and sales, or simulating paper documents.  The remainder of the application is a description of how to deal with the other layers of technology, such as the user interface technology, expression of concepts within the programming language, or interoperability with other applications.  The former is sometimes called essential complexity, the latter accidental complexity.  A math formula doesn't become dated, because it contains minimal "fluff" and maximal "content".  Imagine, however, if every few years we renamed all of the math operators, changed symbols, and required filled pages and pages to define simple equations.  It sounds unproductive, but you could probably sell lots of silly little books with cutesy animal pictures on the front of them to help the poor mathematicians keep up.

The closer we get to systems which describe the essence of their mandate, the less likely they are to become legacy.  In fact, it could be argued that as the accidental complexity approaches zero, it becomes impossible for a system to be dubbed legacy as it is merely a model of reality.  Yes, of course the facet of reality being modeled could change, and so our VHS tape collection tracking software may no longer be needed, but that's not the kind of legacy I'm talking about.  That does lead to a point however.  It is the changing "real world" that spurs our need to change our software models.  Given the information age, the "real world" for most domains is changing faster than ever.  The software abstractions, on the other hand, are more complex than ever, demanding incredible amounts of accidental complexity to achieve anything.  The result:  J2EE, .NET with CAB, LAMP... and a burnt out industry unable to recruit young people because the job is [raise pitch of voice] "boring."

The solution is seemingly simple: minimize accidental complexity.  Easier said than done right?  Actually not really, but [stepping up on soap box] we must take advantage of the "mathematics of data" when modelling data problems.  We must understand that everything we do, at all levels in the software stack, is data management.  Missing this fact is tantamount to trying to model physics using basic arithmetic.  We must define applications first and foremost in their "timeless" logical form, as a relational schema.  I spent the last two days dealing with COM programming.  I'll never get these back.  We must stop thinking that API based abstractions are somehow going to solve a problem; think REST, SOAP, CORBA, DCOM, RPC, etc.  Before I hop off of the soap box, let me vouch for myself.  The prescribed approach DOES work and DOES minimize accidental complexity.  I've now built several applications in Dataphor, a platform that only begins to realize the potential of declarative development, and I've seen that an application can truly be created from little more than defining structures which model reality.  If a system is defined in a high-level, declarative way, let the trendy winds howl as they may, that system will easily evolve, and may not ever need to be replaced.

As a quick disclaimer, I am of course not naive enough to believe that the best technology wins and that politics aren't a primary force in the dynamics of why we are where we are.  Nonetheless, I choose to ignore what I can't control and focus on what can be done in the present.  Also note that a declarative softwaretopia is inherently opposed to "hiding" logic.  This has all sorts of interesting technical ramifications, but it also has some interesting business ramifications.  Perhaps the open source movement is laying useful groundwork.

Posted by Nathan at 11:05 PM | Permalink | Comments (1) | TrackBacks (0)