Professor von Clueless in the Blunder Dome

Republishing before Silence

2010-05-01T00:01:00.001-07:00

The nfoCentrale blogs, including Professor von Clueless, were published through Blogger via FTP transfer to my web sites. That service is ending.

As part of the migration, I am republishing this blog in the latest stable template format.

Then there will be silence as Blogger is unhooked, although the pages will remain.

No new posts or comments will work until I updated the web site to use its own blog engine. Once that migration is completed, posting will resume here, with details about what to know about the transition and any breakage that remains to be repaired.

Meanwhile, if you are curious to watch how this works out, check on Spanner Wingnut’s Muddleware Lab. It may be in various stages of disrepair, but that blog will come under new custodianship first.

Command Line Utilities: What Would Purr Do?

2009-05-12T10:26:00.001-07:00

Technorati Tags: cybersmith, console scripting, Linux Journal, Commandline 101

On the Linux Journal feed, Shawn Powers has a 2005-09-11 Video, “Commandline 101: cat, Not Just for Purring.” Of course, the Unix utility command “cat” is short for concatenate, and knowing that I never think of small furry animals when I see the term in the context of command-line operations. Silly me.

The powers journal is a nice screencast that demonstrates the versatility of this little utility, especially in conjunction with redirection (and also piping, but that probably comes in a later lesson).

Without even watching the tutorial, I was suddenly confronted with the unasked question: What would a utility named purr be good for?

My first thought was creating pleasant audios of purring felines. Then there could be command-line options to control the kind of purr, and even the kind of feline. Do lions purr?

That’s interesting, and perhaps fun, but a little too easy.

A different challenge would be to come up with a legitimate utility function for which purr would become a completely reasonable and recognizable name.

I have no idea.

To confess my advanced state of cluelessness, I also have no immediate ideas about functions named fur, fly, whine, growl, snarl, snore, whimper, etc, ditto, etc.

Retiring InfoNuovo.com

2008-12-31T08:44:00.001-08:00

Technorati Tags: infonuovo.com, InfoNuovo, ODMA, nfoCentrale.com

[cross-posted 2008-12-29T16:42Z from Orcmid’s Lair. Some of the oldest links that still use the infonuovo.com domain are related to ODMA. This post is here to catch those who might end up searching for previously-found ODMA material and wonder where it has gotten too.]

I am retiring the InfoNuovo.com domain after 10 years. The domain will be cast loose at the beginning of February, 2009. Those places where there are still references to infonuovo.com need to be updated:

infonuovo.com, my original “anchor site” for several topical web sites, is now replaced by nfoCentrale.com. From now on there is no reason to refer to either infoNuovo.com or nfoCentrale.com (or its partner, nfoCentrale.net). The topical web sites have their own domain names. Those domain names should be used in bookmarks everywhere.
infonuovo.com/odma will continue to be found at ODMA.info
infonuovo.com/dma will continue to be found at DMAtech.info
infonuovo.com/orcmid will continue to be found at orcmid.com
infonuovo.com/dmware will continue to be found at DMware.info

and so forth for any other infonuovo.com links that may be tucked-away somewhere.

If you have an infonuovo.com bookmark and you are not sure of its replacement, simply use it and notice the URL of the destination that appears in the address bar of your browser. That is the URL that should be bookmarked.

InfoNuovo.com was the first domain name that I ever rented. It was originally hosted on VServers and absorbed through acquisitions a couple of times. On March 22, 1999, I posted my first construction note on the use of InfoNuovo.com as an anchor site, a web site that houses other web sites as part of a single hosting. This was also the first step toward evolution of what I now call the construction structure of any nfoCentrale web site. InfoNuovo was the company name I had chosen for my independent consulting practice initiated on retirement from Xerox Corporation in December, 1998.

When I moved from Silicon Valley to the Seattle Area in August, 1999, I found that InfoNuovo was too easily confused with a name already registered in Washington State. The business became NuovoDoc, but I continued to hold the infonuovo.com domain name for the support of the subwebs housed there. I eventually moved most content to the new anchor, nfoCentrale.net, on Microsoft bCentral.

There was one problem. Although I could redirect unique domain names, such as ODMA.info, to the current anchor, the web pages still served up with the URLs of the actual location on the anchor site. I experimented with URL cloaking, but that created as many problems as it solved.

In October 2006, following the lead of Ed Bott, I switched to A2 Hosting as a way to reduce the hosting fees and also take advantage of the A2 shared hosting Apache-server provisions for addon domains. Addon domains serve up with URLs of their domain even though the domain is anchored on a single hosted site (in this case, nfoCentrale.com). I consolidated all nfoCentrale.net and infonuovo.com content on nfoCentrale.com. I also parked domains nfoCentrale.net and infonuovo.com where they are today, atop nfoCentrale.com. Now, however, accessing any of the individual subwebs triggers redirection to the appropriate addon-domain URL.

This took care of my wanting to have the subwebs always respond as the domains that I have as their addons. It also raised an unexpected problem around case-sensitivity of Apache filenames, a situation I am still digging my way out of. That shows how important having the addon-domain capability is to me. I’m not sure I’d have moved if I knew how difficult the case-sensitivity extrication would be though.

I know that there are still infonuovo.com URLs out there, even though the addon domains have been in place for over two years. In another month, those URLs will fail. I just don’t want to lease infonuovo.com any longer. I do feel a little sentimental about it. That’s not going to stop me.

Confirmable Experience: What a Wideness Gains

2008-10-07T16:16:00.001-07:00

Technorati Tags: confirmable experience, successful communication, dependable systems, usability, cybersmith

[Here’s another confirmable experience cross-posting. There’s some food for thought here for having displays that support your own productivity and enjoyment of sessions at the computer. There’s also something to be cautious about when assuming things about the ways users experience the interfaces that you implement. I know that I often design interfaces for myself, and that may be far short of what is workable for another who doesn’t approach their work in the same way and who doesn’t have the same computer setup.]

Four years ago, I replaced a failing 21” CRT display with a 20” LCD monitor. The improvement was amazing. I have since upgraded my Media Center PC with a graphics card that provided DVI output and there was more improvement.

But the greatest improvement came when the 20” LCD monitor recently began to have morning sickness, flickering on and off for longer and longer times before providing a steady display. Before it failed completely, I began shopping for the best upgrade on the competitive part of the LCD monitor bang-for-buck curve.

These days, 24” widescreen LCD monitors are the bees knees. For almost half what I paid for the 20” LCD in 2004, I obtained a 1920 by 1080 DVI LCD (Dell S2409W) that is not quite the the same 11.75” height but is 21” wide. The visual difference is dramatic when viewing 16:9 format video and also when viewing my now-favorite screensaver. I added a shortcut to my Quick Start toolbar just to be able to watch the screensaver and listen to the bubbles while making notes at my desk.

One of the problems I had with the 20” old-profile (6:4, basically) was that I could not work with multiple documents open at the same time. I don’t mind only having one fully on top, but I often needed to be able to switch between them easily. In some standards-development work that requires comparison of passages in different documents, it was also tricky to have them open in a way where I could line up the material to be compared and checked.

The wider display permits having more of an application open, such as Outlook, and it also allows access to additional open material.

What I hadn’t expected was the tremendous improvement that becomes available when there is a 21” task bar at the bottom of the screen. I did not expect an advantage there as the result of the wider display. That alone has made my working at the computer more enjoyable and more fluid.

My desktop is still too cluttered with icons and I am still tidying them up, removing ones that I rarely use. Even so, the perimeter of the display provides for more icons on the outside of the central work area so that I can find them without having to close or move application windows. That’s another bonus.

I must confess that I haven’t had so much fun since I progressed from Hercules-graphics amber monitors to full-color displays in the early 90s. It is sometimes difficult to realize that it wasn’t that long ago.

Oh Yes, the Confirmable Experience …

There are two confirmable-experience lessons here. First, the subjective experience I am having is mine. The wide-format monitor is an affordance for my heightened excitement and enjoyment, but the experience is mine. Others have different reactions and, in particular, have their own ideas about display real-estate, task bars, and other user-interface provisions.

For the second lesson, recall how much emphasis I give to using a screen-capture utility for computer forensic and trouble-reporting work. That will often provide important out-of-band evidence for a problem that one user is seeing and that another party does not.

These screen captures provide similar evidence of what the wider-format display provides for me.

They don’t provide any assurance that you will see them the same way I do, however. If you click through to the full-size images, you’ll see a rendition of the same bits that my display shows me. I assure you that the image I see when replaying those bits to my screen is exactly the same as the one I took a screen capture of.

There are a number of ways that your experience will be different. At the most fundamental level, there is no way to know, using these images only, to determine whether the color presented for a particular pixel on your display is the same that I see on mine. The PNG files do not reflect what I saw. They do faithfully reflect what my software and graphics card used in the internal image that was presented via my display. But we have no idea whether your computer is presenting the same color using the same bits. There are other differences of course, in that gross features may not be viewable in the same way my monitor allows me to see them (unless yours has at least the 1920 by 1080 resolution that mine does). This is all there to interfere with our sharing this particular experience of mine even without allowance for our different vision and subjectivity influences.

The takeaway for this part is that context matters with regard to what qualifies as a confirmable and confirmed experience. It’s also useful to notice how many different aspects of the computer bits to displayed pixels pipeline can influence whether or not I have successfully shared relevant aspects of my experience with you.

And we do manage to make it all work, most of the time, for most of us.

Confirmable Experience: Consider the Real World

2008-10-05T15:16:00.001-07:00

Technorati Tags: Clarke Ching, confirmable experience, successful communication, dependable systems, trustworthiness, cycle of learning and improvement, usability

[cross-posted from Orcmid’s Lair, essentially for the reasons stated here.]

Clarke Ching just posted a great illustration of a confirmable-experience situation. Until a set of comparative photographs was available to illustrate some different experiences, he and his wife did not know how to understand a difficulty that one had and the other did not (and check the follow-up for more important reality).

This is the entire crux of it.

I often go on about the importance of confirmable experience in the area of trustworthy and dependable systems. Providing confirmable experience is something software producers (and motivated power users) need to pay attention to. Clarke provides the Cool Hand Luke reality version. Sometimes communication is not simple and it is important to remove the barriers.

I posted this on Orcmid’s Lair and I also wanted to drag it into my confirmable-experience cybersmith collection too. I want it there because it is so juicy, even though this is not my main confirmable-experience category location. Well, I think not. I will resolve it for now with cross-posting. Sometimes, I need to make a mess to know that is not the way to do it. Now I have to dig my way out of it.

Cybersmith: IE 8.0 Mitigation #1: Site-wide Compatibility

2008-09-12T23:49:00.001-07:00

Technorati Tags: cybersmith, interoperability, IE8.0 mitigation, web site construction, web standards, compatibility

I have been experimenting with Internet Explorer 8.0 beta 2 enough to realize that all of my own web sites are best viewed in compatibility mode, not standards mode. I find it interesting that other browsers, such as Google Chrome, apparently apply that approach automatically, suggesting to me that the IE 8.0 standards mode is going to cause tremors across the web.

The first step to obtaining immediate, successful viewing under IE 8.0, as well as older and different browsers, is to simply mark all of my sites as requiring compatibility mode. That is the least activity that can possibly work. It provides a tremendous breathing room for being more selective, followed eventually by substitution of fully-standard versions of new and heavily-visited web pages on my sites.

Some pages may remain perpetually under compatibility mode, especially since the convergence of web browsers around HTML 5 support will apparently preserve accommodations for legacy pages designed against non-standard browser behaviors.

This post narrates my effort to accomplish site-wide selection of compatibility mode by making simple changes to web-server parameters, not touching any of the web pages at all.

#0. The Story So Far
#1. The Simplest First Step That Can Possibly Work
2. Satisfying the Prerequisites
3. Experimental Approach for Confirming mod_headers Operation
4. Web Deployment Approach
5. Authoring the .htaccess File
6. Deploying the .htaccess File
7. Confirming .htaccess Success
8. Shampoo, Rinse, Repeat
9. Tools and Resources

See also:
2008-08-30 Orcmid's Lair: Interoperability: The IE 8.0 Disruption for the situation and the basic approach
[undated] MSDN Library: Defining Document Compatibility, Internet Explorer 8.0 beta, preliminary
[undated] MSDN Library: Implementing the META Switch on Apache, preliminary
2008-08-28 Hanu Kommalapati: Apache httpd configuration for IE7 standard mode rendering in IE8 (via Bruce Kyle)

#0. The Story So Far

On installing Internet Explorer 8.0 beta 2, I confirmed that none of my web sites render properly using the default standards-mode rendering. However, my sites render as designed for the past nine years if I view them in compatibility mode.

Although I want to move my high-usage pages to standards-mode over time, I don't want users of Internet 8.0 to have to manually-select compatibility mode when visiting my sites and their blog pages.

What I want now is the simplest step that will advertise to browsers that my pages are all to be viewed in compatibility mode. This will direct the same presentation in IE8.0 as provided by older versions of Internet Explorer and and current browsers (such as Google Chrome) that don't have the IE8.0 standards mode. I can then look at a gradual migration toward having new and high-activity pages be designed for standards-mode viewing while other pages may continue to require compatibility mode indefinitely.

#1. The Simplest First Step That Can Possibly Work

There are ways to have web sites define the required document compatibility without having to touch the existing web pages at all. If I am able to accomplish that, I will have achieved an easy first step:

Have a complete site automatically set to be browsed in compatibility mode (EmulateIE7, in my case), buying time to provide finer grain solutions.

This is accomplished by convincing the web server for my sites to insert the following custom-header line in the headers of every HTTP response that the server makes:

X-UA-Compatible: IE=EmulateIE7

The HTTP-response lines precede the web page that the web server returns. The browser recognizes all lines before the first empty line as headers. Everything following that empty line is the source for the web page. The browser processes the headers it is designed to recognize and ignores any others.

You can see the headers returned as part of an HTTP request by using utilities such as cURL and WFetch. Here are the headers from my primary web site using a show-headers-only request via the command-line tool, cURL:

2. Satisfying the Prerequisites

The MSDN Article on Defining Document Compatibility describes site-wide compatibility control for two web servers: Microsoft Internet Information Server (IIS) and Apache HTTP Server 1.3, 2.0, and 2.2.

Have direct access to the folders of web pages on my web server.
Satisfied: I have FTP access to the complete set of directories that hold web pages for my sites.
Determine that my Apache web server is one of those discussed.
Satisfied: My web-site hosting service uses Apache HTTP Server on Linux. When I check the administrator control panel provided by the hosting service, I discover that I am on Linux kernel 2.6.9 and Apache version 2.2.9 (Unix). The Apache 2.2 requirement is satisfied.
Determine that I can set server-level response.
Not Satisfied: My sites (nfoCentrale.com and all domains that are added-on and shared under it) are on a shared server. I do not have access to the overall server nor to the httpd.conf configuration file.

The Apache 2.2 Module mod_headers documentation and Hanu Kommalapati's example describe how directory-level response headers can be specified instead.

Determine that I can set directory-level response headers.
Not Clear: I know that I can create .htaccess files everywhere on my sites. I don't know whether mod_headers is installed in my server configuration. This list suggests that the option is not available, and there might not even be dynamic loading of extensions (mod_so). The easiest way to confirm the actual state of affairs is by experiment.

To Be Verified: I also submitted a support ticket; the hosting company responded immediately. The answer is, yes, mod_headers is supported on my server. Great! Now to see it working.

3. Experimental Approach for Confirming mod_headers Operation

I want to introduce an .htaccess file that has the following content:

<IfModule headers_module>
Header set X-UA-Compatible: IE=EmulateIE7
</IfModule>
I will try it out on a site that is a placeholder with no meaningful content and no visitors other than myself: eoware.org.
The current state of the site has the following default result when I direct IE8.0 beta2 to http://eoware.org:


This portion of the default page is shown in default standards-mode view. Notice the broken-page button to the left of the refresh button. This indicates that a compatibility-mode view is available.

This rendering is not in agreement with how the page was designed (and not touched since 2006). Notice the color of the horizontal line, and the (added) spacing in the right-justified revision-history information lines.
When compatibility view is selected, the page appears as it was designed (non-standard as it is):



The compatibility button is depressed, the horizontal line is the proper deep-blue color, and the spacing of the right-justified revision-history information is as expected.
The desired state is as in (4) but with no action needed from the user and with the compatibility-view button not presented. This will confirm that, for this site, the EmulateIE7 mode has been specified and automatically honored by the browser. Other browsers (such as the initial Google Chrome release and older versions of Internet Explorer) will default to this view regardless; the custom HTTP header is harmlessly ignored by older browsers.

4. Web Deployment Approach

Having direct FTP access to the web-page folders on my server, along with an out-of-the-way place to try out the change, is relatively safe. Since I am placing an .htaccess file where there presently is none, it is feasible to (1) upload the file, (2) see if it works, and (3) quickly delete it if there is any failure. Having succeeded in introducing .htaccess files for other purposes, I'm confident I can make the change correctly.

I'll not do it that way. Instead, I will rely on my web-deployment-safety model and take advantage of the safety net it affords, even though I could do without it if all I wanted was experimental confirmation. This is a cybersmith post, and I want to illustrate a disciplined approach that has more flexibility in the long run. To see the result that could have been attained by using the direct approach, you can peek ahead to section 6, below, and the image just above it.

Here is the structure of safeguards that I employ to control updates to my sites, keep them backed up, and also have a way to restore/move some or all of the sites. I can also roll-back changes that are incorrect or damaging. I can repair a corrupted site too (and I have had to do that in the past).

The hosted-site web-server pages are maintained using FTP between the server and a private development server which holds a complete image of the hosted-site content. This mirror is in the file system of the development server (an old lap-top running Windows XP and connected on my household LAN for duty as a light-weight server). Synchronization between the site and the mirror is accomplished by FTP in either direction, depending on where new content first appears (on the server for Blogger web logs, on the development server for manually-authored pages and their updates).

**Basic Site Deployment Setup.** The hosted-site is kept synchronized by FTP with a file-system image (left) on a private development server. The content of **http://nfocentrale.com** is mapped to the folder structure starting at **public_html**. The Visual Source Safe project **$/A2HostingWeb** provides source-code management, versioning, and backup of the **public_html** image (right).

In addition to being separately backed-up, the hosted-site image corresponding to http://nfocentrale.com is also the working folder of a Visual SourceSafe project, $/A2HostingWeb. VSS holds the current and also older versions of site material. VSS is also the source of newly-authored material that is ready to be added to the image and published to the hosted-site itself.
At the web-site, the eoware folder under public_html (the http://nfocentrale.com top level) has also been defined as the top level of the eoware.org domain. This is accomplished by use of the hosting-service administrative control panel to create an add-on domain (of an already-leased domain name) at a particular sub-folder of my main site. This feature is a key factor in my choice of the particular hosting service.   In addition, the .htaccess file that I already have in public_html causes any access directly to the eoware/ folder to be redirected to use the http://eoware.org URL. This forces the correct address in the browser for use in book-marking and in search results. These two cURL requests demonstrate the mapping:
To use this deployment path, I need to author the desired .htaccess file and have it checked-into VSS under the $/A2HostingWeb/eoware/ project. To deploy the new material, the content is exported (using Get Latest Version ...) to the file-system site-image location on the development server computer: c:\publicca\A2hosting\public_html\eoware\. Transfer to the web site is by FTP synchronization of the site-image eoware\ folder and the corresponding hosted-site folder.
Although this is a roundabout way to make this individual page, it maintains the practice of not ever authoring directly in the deployment path except under serious emergency. In the absence of an emergency, all development-site authoring occurs elsewhere and is brought under VSS management first. New and changed material flows from VSS to the site image to the web site.

Some new material and changes do originate on the hosted site first. Typical site-first materials include blog posts originated from Blogger and Windows Live Writer, wiki pages (whenever that day comes), and data gathered from web-page forms. Those materials are also periodically synchronized from the hosted-site to the site image and then checked into VSS, keeping a complete site image on the development server under VSS management and for backup.

5. Authoring the .htaccess File

If I was simply making an .htaccess page, I would create it directly in a text editor, having a file such as that created in step 4, below. That page would be saved at a convenient location-machine address and then transferred to the hosted-site using FTP, leading to the result in section 6.

To preserve my development and deployment model, I require more steps.

Although I also have a development web site on the development server, it is an IIS server, not an Apache HTTP Server. The IIS server, FrontPage 2003, and the FrontPage Server Extensions that I use to perform web-page authoring do not permit editing of Apache .htaccess pages and their automatic inclusion under source-code management.

I cling to this web-page authoring model not merely because I have used it since 1998, although that's reason enough. I place high value on the development web-site pages being kept under automatic source-code management using Visual Source Safe. I use the sharing feature of VSS to share the development-site web page content into the $/A2HostingWeb project too. This permits staged synchronization between the development site, the hosted-site image, and indirectly the hosted site itself; and, vice versa. (For those wondering what's on the quiz, I call this a hybrid Microsoft Site Server model, because it works in both directions to also capture authored material that arises initially on the hosted site.)

Because .htaccess files and other Apache-site administration files need to be introduced under source-code management some other way than via FrontPage and my development web site, there's an administrative skeleton project, $/A2HostingAccount, that mimics the folder structure of the hosted-site image just enough to carry any administrative files.

**Working Project for Authoring Adminstrative Files**. Project **$/A2HostingAccount** holds administrative pages that are shared to the hosted-site via **$/A2HostingWeb** (left). Working folders under the local **c:\MyProjects\A2HostingAccount** directory provide check-out and editing of the **.htaccess** file (right).

The $/A2HostingWeb folder structure is replicated in $/A2HostingAccount only to the levels where .htaccess files appear. The same .htaccess file is shared between a folder of $/A2HostingAccount and the same folder of $/A2HostingWeb. In this way, $/A2HostingWeb has source-code management of the entire hosted-site image and $/A2HostingAccount provides a view that is limited to the administrative material.

Because I already have some .htaccess pages for my server, I have lifted an existing one (from nfoworks) by sharing it to eoware, branching it so that changes don't reflect back to nfoworks, and sharing the newly-branched one to $/A2HostingWeb/eoware/ (so that changes will also be seen in that project). This pattern will be re-used to quickly make more .htaccess pages for the root folders of my remaining web sites that need one as part of IE8.0 mitigation.
The customization of the new .htaccess file is accomplished by performing a VSS Get Latest Version ... operation of the eoware project to the working folders on my local machine (on the right, above).

The next step is to open the local .htaccess copy in a VSS-aware editor and check out the file (or check it out first otherwise). The checked-out version is edited to customize it for the new destination, adding the IE8.0-mitigating mod_headers instructions.

After editing is complete, the changed file is saved to disk and checked-in via VSS control.

This is the completed result:

6. Deploying the .htaccess File

Having edited the .htaccess file on my development machine and checked it into VSS (on the development server), the next steps are all conducted on the development server:

Using VSS on the development server, I perform a Get Latest Version ... operation on the $/A2HostingWeb/eoware project. This delivers the newly-customized .htaccess file to the hosted-site image, the working folders for $/A2HostingWeb there.
Using WS_FTP Pro on the development server, I connect to the hosted site.
After connection, I drag the hosted-site image c:\publicca\A2Hosting\public_html\eoware\ folder to the connected-site /public_html folder in the WS_FTP connecting- and connected-site panes. The eoware/ folder of the connected site is updated with any new or more-recent files from the hosted-site image. I see that the .htaccess file is now there.

This is the same deployment procedure for updating of any of my individual sites under the hosted-site. A script for it would be useful. This is on my someday-not-now list. Scripted or not, this is the basic procedure.

7. Confirming .htaccess Success

Assuming that the .htaccess introduction has not derailed the server, confirmation of the parameters and their success is straightforward:

Access to eoware.org with the cURL utility should reveal the custom header in the response. There it is:
Next, access to http://eoware.org/ with IE 8.0 beta 2 should provide a different experience. And here that is:

This access automatically provides the compatibility view and there is no Compatibility View button. (Compare with the second result in section 3.)

8. Shampoo, Rinse, Repeat

We've demonstrated that the .htaccess customization works correctly on my web site and provides the desired result on a little-used URL that is a placeholder for work yet to come. After this cautious effort, it will be straightforward to add similar .htaccess files to each of the individual sites implemented on the hosted-site.

Before that, I will first add the custom-header response to the .htaccess file that is already at public_html, the root of the main site, http://nfocentrale.com. This provides the custom header for all access.

Once all site access returns the custom HTTP header, I can then take my time determining how to work toward migrating sections of web sites to pages that view properly in IE 8.0 standards mode. That will be accounted for as additional mitigation steps.

9. Tools and Resources

The following tools were used in this mitigation step:

cmd.exe, the Microsoft Windows XP command-line console shell
Hypersnap 6, for screen-shot diary and demonstration of the mitigation step
cURL 7.15.4, command-line HTTP request and response capture tool
Apache HTTP Server 2.2.9 (Unix), running as a shared server on a Linux web-hosting system
Windows XP computer for desktop development-site authoring
Windows XP computer providing the development web server (IIS), a co-located source-control database, and the hosted-site image for FTP synchronization with the hosting-service site
Visual SourceSafe 6.0d database on the same computer as the development web server
WS_FTP to synchronize directories without transferring unchanged material
jEdit 4.3 for editing the .htaccess page. Any basic text editor can be used. I use jEdit because (1) I already use it for other things, (2) it integrates with Visual Source Safe, and (3) it provides syntax highlighting for .htaccess files.

DMware: OK, What's CMIS Exactly?

2008-09-10T17:26:00.001-07:00

Technorati Tags: DMware, IBM, Microsoft, CMIS, Content Management, iECM, EMC, Lotus, Sharepoint, Filenet, Open Text, OASIS, Web Services, interoperability, Document Management

There's a nice flurry of interoperability news today, announcing the Content Management Interoperability Services (CMIS) Specification sponsored by EMC, IBM, and Microsoft, with the participation of other content-management vendors, including Open Text. [update: There is extensive coverage on Cover Pages. I recommend that as the comprehensive source.]

Content/Document-Management Integration/Middleware Scheme for This Century?

The stratospheric view from Josh Brodkin suggests that CMIS is a means for cross-over between different content-management regimes as well as bridging from content-aware applications to content-management systems.

The Sharepoint Team describes CMIS as an adapter and integration model for access from content-aware applications in a CMS-neutral way, relying on distributed services via SOAP, REST, and Atom protocols.

The 0.5 draft specification (2008-08-28 6.64MB Zip File download) provides a core data model for expression of managed repository entities, with loosely-coupled interface for application access to repositories via that model:

"The CMIS interface is designed to be layered on top of existing Content Management systems and their existing programmatic interfaces. It is not intended to prescribe how specific features should be implemented within those CM systems, nor to exhaustively expose all of the CM system’s capabilities through the CMIS interfaces. Rather, it is intended to define a generic/universal set of capabilities provided by a CM system and a set of services for working with those capabilities."

It appears that a wide variety of service integrations are possible, although the basic diagram has the familiar shape of an adapter-supported integration on the model of ODBC (and TWAIN and ODMA). Although that's the model, the integration approach is decidedly this-century, relying on relatively-straightforward HTTP-carried protocols rather than client-side integration. Clients must rely on the Service-Oriented Interface, and there is room for provision of client-side adapters to encapsulate that. Either way, this strikes me as timely and very welcome.

Next Steps

The authors have been working for two years to arrive at the draft that will now be submitted to OASIS, estimating that it will take another year to finalize a 1.0 version. [Such a Committee Draft would then go through some rounds of review before promulgation as an OASIS Standard.]

I thought, at first, that this was some form of off-shoot from the AIIM Interoperable ECM (iECM) Standards Project, yet there is no hint of that in the CMIS materials nor on the iECM project and wiki pages.

Announcement of the Proposed TC has just appeared at OASIS [afternoon, September 10]. OASIS Members will make any comments on the proposed charter by September 24, after which there will be a call for participation and then an initial meeting. OASIS members who want to participate in the TC can sign up after the call for participation. The initial meeting is provisionally targeted for a November 10 teleconference. The first face-to-face meeting is planned for three days of mid-January in Redmond. You can follow the charter-discuss list here to see whether there are any questions about the charter, scope, and overlaps with other efforts.

Announcements, Commentary, and Resources

Josh Brodkin: EMC, IBM, Microsoft open up content management interoperability. Storage Knowledge Center, ComputerWorld, 2008-09-10
Larry Cannell: The Commoditization of Content Management? Collaboration and Content Strategies Blog, The Burton Group, 2008-09-10 (via William Vambenepe)
David Choy: Video: CMIS Technical Overview. 2 parts (via YouTube), EMC Labs, 2008-09-09
Robin Cover (ed.): Vendors Publish Content Management Interoperability Services (CMIS) Standard. News Item, Cover Pages, 2008-09-10
Ryan Duguid: Announcing the Content Management Interoperability Services. Microsoft Sharepoint Team Blog, msdn.com, 2008-09-09
David Ferris: CMIS Standard Announced. Ferris Research Blog, 2008-09-09 (via Rob Robinson)
Ethan Gur-esh: Announcing the Content Management Interoperability Services (CMIS) Specification. Microsoft Enterprise Content Management (ECM) Team Blog, msdn.com, 2008-09-09
Chuck Hollis: CMIS -- It's Not JAS. Chuck's Blog, 2008-09-10 (via PatriciaA)
Paul McDougall: Standards Deviant No More? Microsoft Embracing Protocols and Partnerships. InformationWeek, 2008-09-10
Mary McRae: Proposed Charter for OASIS Content Management Interoperability Services. Charter-Discuss List, OASIS Open, 2008-09-10
Gregory Melahn: Content Management Interoperability Services (CMIS). Lotus Quickr wiki, Lotus Software, 2008-09-10 (via IBM Lotus Tech Info)
Microsoft Corporation: EMC, IBM and Microsoft Jointly Create Web Services Interface Specification for Greater Interoperability of Enterprise Content Management Systems. PressPass, microsoft.com, 2008-09-10. Also EMC Newsroom Press Release, 2008-09-10; IBM Corporation Content Management Interoperability Services (CMIS) page (undated).
John Newton: Alfresco releases first CMIS implementation. Content Log, 2008-09-10 (via William Vambenepe)
Open Text Corporation: Open Text Pledges Support for New Content Management Interoperability Standard (CMIS), PR Newswire, 2008-09-10 (via Charged)
Craig Randall: CMIS - Content Management Interoperability Services. Craig's Musings, 2008-09-10 (via PatriciaA)
Craig Randall: Forrester's Kyle McNabb discusses Content Management Interoperability Services (CMIS). 14m30s podcast (MP3), undated.
Jignesh Shaw: Video: CMIS Application Scenarios. (6m11s via YouTube), EMC Labs, 2008-09-09
Jeff Teper: CMIS Specification (3m11s WMV video), Content Management Interoperability Services (CMIS) Virtual Pressroom, PressPass, microsoft.com, 2008-09-10
William Vambenepe: CMIS, APP, Zen-SOAP and WS-KitchenSink: some data points. William Vambenepe's blog: IT management in a changing IT world, 2008-09-11

[update 2008-09-12T08:09Z I am having trouble getting Blogger to push updates through FTP to my site. This repost is an attempt to get the previous changes posted. update 2008-09-11T19:12Z Well, added some interesting links as deeper analysis and pontification arises. I don't expect to add more unless Dare or Tim Bray chime in. update 2008-09-11T15:43Z Use full-size CMIS diagram from EMC (via Cover Pages) update 2008-09-11T15:35Z Repair handling of images and attract attention to the Service-Oriented Interface notion employed in the CMIS diagram. update 2008-09-11T15:17Z Add link to comprehensive Cover Pages compilation. I'm also praying that Blogger's FTP update succeeds sometime in the proximity to my submitting the post. update 2008-09-11T02:37Z Add links to additional resources at EMC and to add images/videos to the page. update 2008-09-11T01:15Z Added more links and information about the OASIS Proposed Charter for the CMIS TC.]

Document Interoperability: The Web Lesson

2008-09-07T17:25:00.001-07:00

Technorati Tags: interoperability, IE8, web site construction, web standards, compatibility, conformance, document preservation, document formats, IE8.0 mitigation, HTML 5

[update 2008-09-08T00:24Z Cross-posted from Pursuing Harmony because of the overlap with convergence of HTML, web standards, and the IE80.0 mitigation that is touched on here.]

"are there alternatives to google groups search for searching old USENET messages? because groups date fielded search is teh broken."

-- Richard Akerman on Twitter, 2008-08-31

Be prepared for a dramatic shift in the reality of web-site browsing and the honoring of web-page standards. The pending release of Microsoft Internet Explorer 8 is going to put the reality of web standards and their loose adherence in our faces. Although Internet Explorer is indicted as the archetypical contributor to disharmony on the web, Internet Explorer 8 is going to challenge all of us to deal with the reality of our mutual contribution to the current state of affairs.

Here is a lesson, probably many lessons, for document interoperability and the way that standards for document formats evolve and harmonize, or not, over time.

The Web as Clinical Science

The movement from loosely-standard pages and their browsing to strictly-standard pages and standards-mode browsing will illustrate every aspect of the same challenge for office-productivity documents and the office suites that process them.

Web pages are the experimental drosophilae of digital documents. All aspects of dynamic convergence on standards, themselves evolving, and the forces of divergence, are demonstrated clearly and rapidly. I expect it to take Internet generations for significant convergence, with no static level of standards adherence anywhere in sight. It took us almost 20 years to get to this point on the Web; I figure it will take at least five more to dig out of it far enough to claim that there is a standards-based web in existence and in practice. I'm optimistic, considering that HTML 5, the great stabilization, is not expected to achieve W3C Recommendation status until 2012.

No document-interoperability convergence effort is anywhere close to the promising situation of the web as Internet Explorer 8, HTML5 implementations, and other compatibility-savvy browsers roll out over the next several years. It is useful to use that situation to calibrate how convergence and interoperability could work for document interoperability. There are significant technical barriers. The non-technical barriers are the most daunting. That should be no surprise.

Versioning in Document Use

I've written on Orcmid's Lair about the IE 8.0 Disruption. This involves changes in Internet Explorer 8.0 by which web pages are rendered in standards-mode on the assumption that pages are conformant with applicable web standards. In the past, it was presumed that pages were loosely-standard and browsers, also loosely-standard, made a kind of best effort to present the page. The consequences have been explained marvelously in Joel Spolski's post on Martian Headsets.

We are similarly relying on document-format standards as a way to provide for many-to-many interchange and interoperability between different (implementations of versions of) document-format standards and different (implementations of versions of) processors of those digital documents. That means we have a version of the loosely-standard documents with loosely-standard processing problem. We can't be strictly standard because the standards can't (and definitely don't) have strict implementations at the moment; and there are many ways that specifications and implementations have been kept loose by design. Accompanying that looseness by design is the the simple fact of immaturity among the contending document-format standards for office applications, particularly as vehicles for interoperable applications.

For office-productivity documents as we know and love them, there are five, count 'em five "official standards."

The "Official" Public Standards of Office Documents

For Office Open XML Format (OOXML), there is the ECMA-376 specification of December 2006. There is also the ISO/IEC 29500:2008 Office Open XML File Formats standard once it is made available. IS 29500 will have some substantive differences from ECMA-376. We won't have a solid calibration of the differences until the IS 29500 specifications are available and subject to extensive review.

For the OpenDocument Format, there is the Open Document Format for Office Applications (OpenDocument) v1.0 OASIS Standard issued 1 May 2005. There is also the ISO/IEC 26300:2006 Open Document For Office Applications (OpenDocument) v1.0 standard (also on the publicly-available listing). IS 26300 is for the same format as the OASIS v1.0 standard, but it is on a completely-separate standards progression. Appendix E.3 accounts for the differences of IS 26300 from the text of the May 2005 OASIS Standard. The first page of the IS 26300:2006 document (page 5 of the PDF) identifies its source as Open Document Format for Office Applications (OpenDocument) v1.0 (Second Edition) Committee Specification 1, dated 19 July 2006, derived from document file OpenDocument-v1.0ed2-cs1.odt; this is not another OASIS Standard, however.

The second and latest OASIS Standard for ODF is Open Document Format for Office Applications (OpenDocument) v1.1 issued 2 February 2007. This document is derived from OpenDocument v1.0 (Second Edition) Committee Specification 1, the same specification that is the source of content for ISO/IEC 26300:2006. The changes made to arrive at ODF v1.1 from the v1.0 (Second Edition) committee specification are detailed in Appendix G.4. There are some mildly-breaking changes from ODF v1.0 to ODF v1.1, mostly of a clarification or correction nature. There are a few additional features that have no down-level counterparts in ODF v1.0.

A third OASIS Standard, ODF v1.2, is under development. The current drafts, using a very-different organization from v1.1, are available as pubic documents of the OASIS Open Document TC.

We can expect to see more versions of ODF and of OOXML at their various standards venues. We'll be watching here on nfoWorks as the situation becomes even more chaotic. Notice that this diversity ignores the variety of divergent implementations of the various specifications.

Format Versions that Live Forever

It is possible for one document-format specification to officially supplant another, with the older specification deprecated. That has not been done so far with any of the five-and-growing document-format specifications, any more than it has been done for most of the versions of HTML specifications that have been recommendations of the W3C (and IETF before the development track moved entirely to W3C).

For example, the last full-up specification for HTML, the HTML 4.01 W3C Recommendation of 24 December 1999, has this to say about its immediate predecessor: "This document obsoletes previous versions of HTML 4.0, although W3C will continue to make those specifications and their DTDs available at the W3C Web site." This was possible because HTML 4.0 was young and there were important defects that 4.01 cured.

The HTML 4.01 specification continues with the following recommendation: "W3C recommends that user agents and authors (and in particular, authoring tools) produce HTML 4.01 documents rather than HTML 4.0 documents. W3C recommends that authors produce HTML 4 documents instead of HTML 3.2 documents. For reasons of backward compatibility, W3C also recommends that tools interpreting HTML 4 continue to support HTML 3.2 [W3C Recommendation 14 January 1997] and HTML 2.0 [IETF rfc1866 November 1995 and the IETF-obsoleting rfc2854 June 2000] as well."

The XHTML branch of specifications, originally derived from HTML 4.01, were intended as the basis for a future generation.

Meanwhile, there has been work toward both XHTML 2 and HTML 5.0.

HTML 5.0 is currently intended to exist alongside XHTML 1.x and its newer arrangements while also absorbing XHTML 1.x to some degree (by having an XML form). The current HTML 5.0 draft specifies legacy processing (in its HTML-syntax form) for variations of over 60 HTML DOCTYPE DTD flavors, extending back to HTML 1.0 and other variants. The intention is to converge HTML and XHTML 1.x under a consistent HTML 5 processing model with only no-quirks, some-quirks, and quirks modes. This is also intended to end the variation and extension of HTML (not XHTML) by capturing <!DOCTYPE HTML> for its own and having a concrete HTML syntax that is fully-divorced from both SGML and XML. It is important to point out that HTML 5 is not going to eliminate the divergence that browser (user-agent) plug-in models, plug-in implementations and scripting systems (especially client side) bring to the mix.

Document-format versions are not easily abandoned. Even if production of a format is deprecated, consumption of the format may need to continue into the indefinite future, and certainly so long as emitters of deprecated formats have significant usage. The W3C progression of HTML is at a point where that is fully-recognized and being honored in reaching toward an HTML 5 plateau sometime in the next decade.

Considering this promising stabilization, when would I manage to change all of my web sites and blogs to clean HTML 5 pages? Not until I know that visits to those sites are only a small fraction of Internet Explorer versions prior to IE8 (or maybe IE9) and other browsers lacking full-up standards-mode processing. Fortunately, the HTML 5 specification-effort promises to show me exactly how to do that in a mechanical way. I am looking forward to automated assistance. In my case, I'll also have the benefit of my IE 8.0 mitigation effort. Other web sites may require other approaches, and user browser choice will involve important trade-offs for some time.

I am surprised by the number of people who operate multiple browsers. Although I operate multiple products for office applications these days, that's mostly to explore their interoperable use, not to ensure ability to interchange documents (well, not until I joined OASIS and the ODF TC). I've been a serial adopter of Internet Explorer versions since IE 2.0. As a typical late-adopter, I may finally branch out now just to have a better calibration of the migration to standards-based sites and browsers for them.

This is an important lesson for the management of the expanding variety of specifications of formats for office-application documents, formats of which HTML packagings are sometimes one of the flavors.

Reconciling office-application document-format versions does not promise to be so easy as the current effort to stabilize HTML for the web.

The Looseness of Document Specifications

Of course, OOXML and ODF are not close dialects off a single family tree, as HTML variants might be treated (and HTML 5 demonstrates, if successful). In addition, the current specifications are not for same-conformance, interchangeable-everywhere documents:

There are weak conformance requirements. It is not necessary to implement any particular amount of the specified format: OOXML or ODF. This is by design. I don't expect that to change. There is also no way to indicate how much or how little is accepted and/or produced. Well, you could look to see what software produced the document, using ODF as our example:

<office:document-meta
    xmlns:office="urn:oasis:names:tc:opendocument:xmlns:office:1.0"
    xmlns:meta="urn:oasis:names:tc:opendocument:xmlns:meta:1.0"
    office:version="1.2">
<office:meta>
    <meta:generator>
        OpenOffice.org/3.0_Beta$Win32 OpenOffice.org_project/300m3$Build-9328
    </meta:generator>
</office:meta>
</office:document-meta>

This strikes me as even less appealing than the challenge of sites adjusting for browsers and browsers adjusting to HTML DOCTYPE declarations (and their absence).

It is not encouraging that the office:version attribute and <meta:generator> element are both optional. It is unfortunate that the office:version attribute is generally uninformative about the processing requirements for the document file in hand, serving merely as an automatic claim of one specification the document conforms to. The document is also likely to conform to earlier versions and probably ~~alter~~ later versions, although it is unclear how we can determine that easily for a given document representation.

Arbitrary "foreign" elements are allowed. I'm not clear how IS 29500 for OOXML will allow for this kind of thing, but the ODF specifications are justly-notorious for this provision (ODF 1.1, section 1.5):

"Documents that conform to the OpenDocument specification may contain elements and attributes not specified within the OpenDocument schema. Such elements and attributes must not be part of a namespace that is defined within this specification and are called foreign elements and attributes.

"Conforming applications either shall read documents that are valid against the OpenDocument schema if all foreign elements and attributes are removed before validation takes place, or shall write documents that are valid against the OpenDocument schema if all foreign elements and attributes are removed before validation takes place.

"Conforming applications that read and write documents may preserve foreign elements and attributes."

There are some further wrinkles and this proviso:

"Foreign elements may have an office:process-content attribute attached that has the value true or false. If the attribute's value is true, or if the attribute does not exist, the element's content should be processed by conforming applications. Otherwise conforming applications should not process the element's content, but may only preserve its content. If the element's content should be processed, the document itself shall be valid against the OpenDocument schema if the unknown element is replaced with its content only."

As a developer, I love gimmicks like this. But, basically, this only works with processors that re-encounter document files that they themselves produced. Anything more coherent requires that the implementers of different processors form some sort of out-of-band, separate-from-the-standard interoperability agreement on particular foreign elements and handling of office:process-content attributes. Users, confident that their software is "standard," will have frustrating and inexplicable interchange experiences (unless the usual thing is done and everyone agrees to lock in on the same software [version], surprise, surprise).
OOXML has a versioning scheme that might provide controlled extensions that degrade usefully when processed by implementations of down-level specification versions. It is unclear at this point whether this is just a more complicated way to end up with the same interoperability problems.
Some features require foreign content. Both OOXML and ODF have features where content is represented by a binary-data part elsewhere in the package. There is little (OOXML) or no (ODF) indication of what the format of the binary element is and what MIME types are allowed for such document components. All use of those features and any interchange agreements about them are beyond the current provisions of the relevant document-format standards.

There are other places where implementation-defined values are expected and are expected to be preserved by other implementations.
Some values and default selections are implementation-specific. I was mining in the ODF specification the other day. I did not expect to find attributes having text on these patterns:

"The value of this attribute is implementation [or application] specific."

"If this attribute is not present, the application might or might not display [whatever]."

These are relatively minor considering the amount of variability from the other conditions already mentioned. What's curious about these is the elevation of particular implementation-specific features as specification-favored. In the case of implementation-specific attribute values, there is also the interesting problem of a processor determining whether such a value is intended to have its implementation-specific interpretation or not. It appears that the related features will only be useful under tightly-restricted interchange conditions.

I will not be surprised to find similar looseness in the OOXML specification, IS 29500.

Prospects for Interoperable Convergence

We already have before us difficulties with interoperable convergence of individual progression of a single standard and its variety of implementation. This makes the prospect of harmonization between different standard formats rather murky.

Desktop office-application software has more promise with regard to application of Postel's Law, to be liberal in what is accepted and conservative in what is produced. Unfortunately, the current specifications do not require conservative, interoperable implementations; the current specifications are arguably antagonistic to such an achievement.

I suspect that this is an unintended consequence mixed with some inattention to what it takes for interoperability to be achievable.

It remains to see how our experience and understanding matures. We are at the beginning, not the finish. The journey may seem endless.

The process of IE 8.0 mitigation and preparation for a standards-mode approach to web browsing impacts this site and blog as well as every other web page I have ever posted (somewhere over 120MB worth and climbing).

I'm not going to say anything more about IE 8.0 mitigation and HTML harmonization here. The overall effort will be tracked in that category of Professor von Clueless posts; that's the place to follow along. The lesson for document interoperability is something that is definitely appropriate for Pursuing Harmony; there'll be much more to say about that.

Cybersmith: The IE 8.0 Disruption

2008-09-02T20:07:00.001-07:00

Technorati Tags: interoperability, web standards, trustworthiness, IE8, usability, web site construction, compatibility, cybersmith

[2008-09-03T03:04Z cross-posted from Orcmid's Lair, capturing the post under the IE8.0 mitigation category here.]

I've elected to adopt the IE 8.0 beta 2 release as a tool for checking the compatibility of web and blog pages of mine. I see how disruptive the change to default standards-mode is going to be and how IE 8.0 is going to assist us. I need to dig out tools and resources that will help me mitigate the disruption and end up with standards-compliant pages as the default for new pages.

Looking Over IE 8.0 beta 2

I avoid beta releases of desk-top software, including operating systems and browsers. Because the standards-mode default of IE 8.0 is going to place significant demands on web sites, I also thought it time to install one copy of IE 8.0 simply to begin assessing all of my web sites and blog pages for being standard-compliant enough to get by. I am willing to risk use of beta-level software in order to be prepared for the official release in this specific case. I'm also sick of having IE 7.0 hang and crash on mundane pages such as my amazon.com logon. I'm hoping that even the beta of IE 8.0 will give me some relief from the IE 7.0 unreliability experience. And so far, so good.

With the promotion of beta2 downloading this past week, I took the plunge. Installation was uneventful and all of my settings, add-ins, favorites and history were preserved. My existing home page, default selections, menus and tool bars were also preserved. [I am using Windows XP SP3 on a Windows Media Center PC purchased in September, 2005. IE 8.0 beta 2 also seems faster on this system in all of its modes.]

I did not review much of the information available on IE 8.0, expecting to simply try it out.

My first surprise was a change to the address bar. There is a new format where all but the domain name of the URL are grayed. That was distracting for the first few days and it still has me stop and think. I realized this is the point: emphasizing the domain name so that people will tend to check whether they are where they expect to be. I like the idea, even though I have to look carefully and remember the full URL is there when I want to paste it somewhere or share the page on FriendFeed or elsewhere. I take this provision as one of those small details that demonstrates a commitment to safe browsing and confident use of the Internet.

What I was looking for, and saw immediately, is the new compatibility-view button. This "broken page" button appeared on the first site I visited after installation of IE 8.0 beta 2.

Clicking the button causes it to be shown as depressed and the page is re-rendered as a loosely-standard page with the best-effort presentation and quirks renderings of IE 7.0 and earlier Internet Explorer releases. If you leave the button selected, the setting is remembered and automatically-selected on your next visits to the same domain. It stays that way until you unselect the button by clicking it again while visiting pages of that domain. It was this feature that tipped-me over in wanting to check out my own pages using beta2 (although I thought the button was tracked at the individual page level until I read the description of domain-level setting).

By the way, if a page is detected to require a standards or compatibility mode specifically, no compatibility view option button is presented.The amazon.com site is this way from my computer, and so is Vicki's pottery-site home page. I looked at the source of the amazon.com site and confirmed that they are not using the special tag that requests that the compatibility view be automatic. I didn't check the HTTP headers to see if they are using that approach to forcing a compatibility or a standards-mode view. I know I did nothing of the kind on Vicki's site. This suggests to me that there is also some filtering going on in standards-mode rendering to notice whether a compatibility view should be offered. I'm baffled here. I am curious whether there is any browser indication when the compatibility view is selected by a web page tag or HTTP header. I suspect not and I'll have checked into that soon enough.

I also checked out the InPrivate browsing feature, which, although popularly dubbed the "porn mode," is very useful when using a browser from a kiosk or Internet cafe and when making private on-line transactions from home.

At this point, I am not interested in special features of IE 8.0 other than those related to improving the standards-compliant qualities of web pages and the browsing experience. I may experiment with other features later. My primary objective is to use the facilities of IE 8.0 and accompanying tools to improve the quality and longevity of my web publications. Once I have some mastery over web standards, I will look into accessibility considerations, another project I have been avoiding.

Disrupting the State of the Web

The problem that IE 8.0 is intended to help resolve is the abuse of Postel's Law [compatibility view offered] that the web represents: "be conservative in what you do, be liberal in what you accept from others." The abuse arises when what you do is based on what is being accepted, with no idea what it means to be conservative. The web was and is an HTML Wild West and it is very difficult to enforce conservatism (that is, strict standards conformance in web-page creation). Since browsers also varied in what they accepted and then what they did with it, loosely-standard pages and loosely-standard browsers have been the norm and web pages are crafted to match up with the actual response of popular browsers.

Since Internet Explorer is made the heavy in this story, we now get to see the price of changing over to "be strict in what is accepted and be standard in what is done with it." This is a very disruptive change. We'll see how well it works. Joe Gregorio argues that exceptions to Postel's Law are appropriate. Some, like Joel Spolski [no compatibility view], think it might be a little too late. There are already some who claim that the IE 8.0 Compatibility view is a sin against standardization [compatibility view offered], no matter that not many of the 8 billion and climbing pages out there are going to be made strictly-conformant any time soon. With regard to compatibility mode, I think it is foolish for it not to be there and Mary-Jo Foley is correct to wonder how much complainers are grasping at straws.

It was surprising to me to observe how regularly the compatibility-view option button appears and how terribly much of my material renders in IE 8.0's standards mode. Apparently the button is there because IE 8.0 can't tell whether the page is really meant to be rendered via standards-mode or is actually a loosely-implemented page. I'm spending a fair amount of time toggling back and forth to see if there is any difference on sites I visit. This suggests to me that there is going to be a rude awakening everywhere real soon now. It is also clear to me that I don't fully understand exactly how this works, and I need to find a way to test the explanation on the IE blog and the discrepancies I notice, especially when the compatibility-view option is not offered and I know nothing special was done to accomplish that on the web page I am visiting. I am also getting conflicting advice when I use an on-line web-page validator.

This change-over to unforgiving, default-standards-mode browsers is going to be very disruptive for the Internet. In many cases, especially for older, not-actively-maintained material, the compatibility view is the only way to continue to access the material successfully. There is a great deal of material for which it is either too expensive or flatly inappropriate to re-format for compatible rendering using strictly-standard features. Without compatibility view, I don't think a transition to standards mode could be possible. The feature strikes me as a brilliant approach to a very sticky situation.

Although there is a way to identify individual pages as being loosely-standard and intended for automatic compatibility view, that still means the pages have to be touched and replaced, even to add one line to the <head> element of the HTML page. There are billions of pages that may require that treatment. Perhaps many of them will be adjusted. That will take time. Meanwhile, having the compatibility-view option and its automatic presentation is very important.

There is also a way to adjust a web server to provide HTML headers that request a compatibility (or standards-mode only) view of all pages from a given domain. That strikes me as a desperate option to be used only when there is no intention of repairing pages of the site. I might do that temporarily, but only while I am preparing for a more-constructive solution that doesn't depend on compatibility view being supported into the indefinite future. The variations on the available forms of control (browser mode, DOCTYPE, HTTP header, and meta-tag) need to be studied carefully. I expect there to be confusion for a while, probably because I am feeling confused with the ambiguities in my experience so far.

Another problem, especially with regard to IE 8.0 beta2, is that we don't reliably know how badly a loosely-standard page will render with a final standards-mode browser versus the terrible standards-mode rendering that beta2 sometimes makes at this time. It is conceivable that the degradation might not be quite so bad as it appears in beta2, but there is no way to tell just yet.

The need for expertise and facility with semi-automated tools as part of preserving sites with standards-conforming web pages is probably a short-term business opportunity. The web sites that may be able to make the transition most easily may be those like Wikipedia, where the pages are generated from non-HTML source material. (That makes it surprising that Wikipedia pages currently provoke compatibility buttons and compatibility view is needed to do simple things like be able to follow links in an article's outline.)

Mitigating IE 8.0

To mitigate the impact of IE 8.0 becoming heavily used, it is necessary to find ways to do the least that can possibly work at once, and then to apply that same attitude in making the next most-useful change, and so on, until the desired mix of standards-compliant and loosely-compliant pages is achieved.

To find out what tools are available along with IE8 beta 2, these pages provide some great guidance and resources:

Ed Bott 2008-08-28: An IE8 Beta 2 Q&A
IE8 Blog 2008-08-27: Internet Explorer 8 Beta 2 Now Available
Microsoft: Windows Internet Explorer 8 (beta): Home Page
US ISV Developer Evangelism Team: Consumers Begin Using Internet Explorer 8 Beta 2
MSDN online: META Tags and Locking in Future Compatibility (preliminary), with details for how to mark pages and also set the sites HTTP response headers (not for the faint-hearted)

That should point you to all of the resources you need to understand how to check sites, how to use the compatibility provisions, and other ways to take advantage of IE8 availability when it exits beta.

I'm looking at a progression that will allow the following:

Have a complete site automatically set to be browsed in compatibility mode (EmulateIE7, in my case), buying time to provide finer grain solutions
Modify templates on blogs such as this one to specify compatibility mode on all new and updated pages until I say otherwise
Find a way to make bulk changes to pages, adding a <meta> head element that specifies compatibility mode for those pages
Decide how to migrate pages so that their results are delivered best in standards mode. This may be a very long-term approach that doesn't begin implementation until the percentage of old browsers still in use diminishes enough to have standards-mode browsers be dominant. There should still be a substantial period of time while compatibility mode is grand-fathered by the latest browsers.
Even if compatibility modes eventually disappear from popular browsers or whatever comes after the browser, there will be a lasting need for compatibility view of archival materials, or some other creative solution that allows those materials to be accessed in a standards-mode world.
There will be future breaking changes in standards mode as updated/successor standards are introduced. The compatibility view requirement may never disappear, although its future achievement may be accomplished with less disruption. Unless, of course, we fail to learn the lesson.

I will work out my own approach on Professor von Clueless, since I have definitely blundered my way into this.

This post is also being used to identify the IE8 mitigation required for this blog, along with some other improvements:

It is a prospect for assessment of standards-compliant presentation via IE8 beta 2.
It is my first use of Blogger Labels (Categories in other blogging systems) to archive Orcmid's Lair posts by categories as well as having weekly chronological archive pages and the buckets of individual posts made in the same month.
I am setting up Windows Live Writer image-uploading capability to FTP images to a directory of the blog; the images in this post are the confirmation of that.

When I update the template to force compatibility with the current loosely-standard blog-page generation, this post will reflect that too.

[update 2008-08-30T16:42Z I had a few clumsy bits to clean up, taking the opportunity to elaborate further in some areas. The disruption with standards-mode web browsing is a great lesson for standards-based document-processing systems and office-suite migrations toward document interoperability. I'm going to pay attention to that from the perspective of the Harmony Principles too.]

Cybersmith: The Confirmability of Confirmable Experience

2008-08-29T20:20:00.001-07:00

Technorati Tags: cybersmith, confirmable experience, interoperability, trustworthiness, IE8, screen capture, usability, web site construction

Finding ways for the experience of users to be confirmable by the producers of software is increasingly difficult as we operate with distributed applications over networks and the world-wide web. Because we can't directly show another user or the software producer what our experience is, we need forensic tools that allow us to capture and communicate the locally-observed behavior to others who are elsewhere. I always keep screen capture software handy. An experience with the new Internet Explorer 8 beta 2 release demonstrates the value of that.

Screen Capture: the Primo Confirmability Utility

One of the most-important tools for cybersmiths, including power users, is a screen-capture utility. Whenever I set up a new computer, my favorite screen capture utility (currently HyperSnap 6.30) is one of the first two products I install. (The other is WinZip for its value in addition to the built-in Zip capability of Windows Explorer. That's actually in a three-way tie with my password-safe utility and Microsoft OneCare.)

If I could count on a screen saver being available during initial set-up (even log-on if that were possible) and configuration of a new computer's operating system, I would be even happier. I want a screen-shot record of everything that I go through and of every option and setting and parameter that I choose. I do the same thing whenever I am installing a new software package for the first few times.

And whenever there is an unusual incident, I start grabbing screen shots as long as I am able. If I can't make screen captures, I will grab my digital camera or (though needing to get the hang of it still) my Windows Mobile cellular phone.

Although there is limited screen capture capability built into systems like Windows, I rarely want the entire screen. Also, I want to save in a loss-less compact format, almost always preferring PNG format. This format is easily included in e-mails and posted on a web site to back up an incident report or provide documentation of something interesting. All of the VC++ Novice screen shots have been created this way.

Screen Capture: Do You See What I See?

In today's world of distributed applications, one of the greatest difficulties is dealing with interoperability problems (e.g., garbled e-mail messages, broken web-page presentations, and document viewing/presentation glitches). Even if an offending file or document is sent back to the source with an incident description, the recipient may not see what you saw. Reproduction isn't even the first problem. Clearly seeing what it is you experienced is the first problem. There needs to be an out-of-the-failing-channel way to deliver a visible rendition of what you are seeing. Sending a screen shot will do that.

Confirmable Experience for Trustworthiness

This ability to report your experience in a way that a distant party can confirm it is a critical need in today's richly-variable and richly-connected world. I expect that we will eventually rate software and its support by how well it lends itself to confirmation of user experiences, along with easy reproducibility and remedy of unexpected situations and usability difficulties. The first step is to be able to demonstrate what happened in a simple way. Today, use of screen shots is one of the easiest and reliable ways to do that. There often needs to be an accompanying narrative and any data or files involved, but the screen captures are essential for clarity.

My colleague Bill Anderson and I use the term "confirmable experience" almost habitually when discussing interoperability breakdowns, inscrutable installation instructions, and the occasional flagrant exhibition of system incoherence.

There's another kind of confirmable experience, and it is the one I can avail myself of as a developer and software producer. There are many cases where I need to find ways to confirm what kind of experience my software products (including web pages) are providing and what the breakdowns are. Screen capture is useful for documentation and demonstrations as well as being an important forensic tool.

An Example with Internet Explorer 8.0

I am one of those people who provides an absolute minimum of testing for web sites and blog pages. I figure that if the material shows properly in my authoring tools and also in my usual browser (yes, Internet Explorer), I am on safe ground. Also, because I do not use scripts and I have static pages almost entirely (with the occasional embedded video frame), I rely on pretty-simple HTML.

I also figure that I am fairly safe from problems with standards-strict browsers of various flavors. I make sure that my recent web pages are in UTF-8 (to minimize character encoding issues) and have an unassuming DOCTYPE declaration (for HTML 4.01 transitional mainly).

I've never received a report about rendering difficulties with my web pages, although that's not a reliable test.

Knowing that Internet Explorer 8.0 is going to usher in a period of widespread standards-strictness, I figured that I was still relatively safe. Just the same, when IE 8.0 beta 2 was released the other day, I decided it was time that I took my chances with the browser in order to find out how well my sites will be viewed (expecting an immediate "pass" grade of course). Another value of confirmable experiences is separation of fantasy and magical thinking from reality. Sure enough ...

Once I had installed IE8 beta 2, my first check was with my default home page, http://my.yahoo.com, where I regularly check the weather, headlines and movie listings for my favorite local cinema. The page passed the eyeball test, although I now see that there is a ~~broken-compatibility indicator~~ compatibility-view button. The indicator really does disappear when a page is ~~"clean,"~~ specifically selected in compatability view or standards-mode, as I can see on some other yahoo.com pages. If there is an indicator and I push it in (it is a push-on, push-again-for-off button), I can see minor changes on that home page. This is exactly what I want for a site-forensic tool, and IE8 makes it easy for me to review my site, find the discrepancies and, if serious enough, repair them. There is also a third case, where I add metadata to pages or an entire site so they will be automatically-recognized ~~as non-standard~~ under compatibility view and treated in a forgiving way by IE8 the same way its predecessor had by default. Finally, users of IE8 can select compatibility with loosely-standard pages themselves, and IE8 will remember that selection on a ~~page-by-page~~ domain basis.

**Illustration of IE8 beta 2 Compatibility Control:** (a, left) strict presentation with compatibility defects, (b, center) rendering with relaxed rules, (c, right) absence of indicator with fully-standard page

Notice, also, that the experience I had is with my my.yahoo.com, not yours or anyone else's. This is an easy way to demonstrate to someone what my experience is, no matter what their own checking reveals. I love how this is working out. And of course, my simpler pages would fare well. Sure ...

Not So Fast, Sparky

One of the oldest web pages that I still have on a site is the home page for Orcmid's Liar, http://orcmid.com. This is a trivial page, essentially a place-holder for riches yet to come, while I distract myself with blogging, other content, and little concern for this site's puny entrance. I expected this to be a no-brainer.

**Whoops:** Presentation of my home page in IE8 beta2 strict-standards mode is unbelievably ugly.

The result is terrible. The single-row, three-column table at the top of the page is mangled. There is text missing from the central cell and the table does not span across the width of the page as intended. It looks like the table has been wrapped on the page with the third cell below the first two.

Compatibility mode (that is, loose enforcement of standards with a best-efforts rendering) provides what I am accustomed too:

**Expected result:** Viewing under the forgiving compatibility mode provides the expected rendition.

This is a common page top-heading format used throughout my web sites. I nosed around to see if they all render so badly in IE8 ~~strict~~ standards mode. There are incompatibilities, but not the one seen on the Orcmid's Lair home page.

While wandering around, I noticed another problem. Sometimes when I return to the home page, viewed in ~~strict~~ standards mode, by back-paging from a different page, I see a rather different incorrect version. This one is peculiarly almost-right:

**Erratic strictness:** If the page is visited, navigated through, and then back-paged back, the strict rendering is different.

Exactly Whose Experience Are We Experiencing?

Granted, my home page is not strictly-standard according to IE8 beta2, and it is presented properly only with the forgiving compatibility mode for loosely-standard HTML. At the same time, absence of consistent behavior is an indication that there may still be defects in the IE8 implementation of strict-presentation in accordance with web standards.

I was still left with the problem of straightening this out. It seemed to me that the page is simple enough that I could repair the page and not resort to the special meta-tag to request compatibility mode in browsers, such as IE8, that will recognize and automatically accept loosely-standard pages and render them as well as IE7 does.

I attempted to edit the HTML, making adjustments to the 3-column table that I though might help. There was no change. But there are other pages on the site with the same 3-piece top heading and those render fine. So I copied one of the successful 3-column tables in place of the current one, edited the text appropriately, and found success:

**That's more like it:** The repaired table, shown in this narrower view, is presented just fine in IE8 beta2's strict-standards processing.

The compatibility button has not disappeared, and there is more to accomplish. But the ugliest part has been repaired successfully. I'm not sure what the difference was, so there is more to learn.

The next action, for me, is to use an HTML validator that lets me attest to the validity of the pages on my sites. That will come later as I find occasion to review all of the pages, treating the most-embarrassing glitches first.

I also must deal with the fact that all I have done so far is treat symptoms, without a clue regarding the underlying cause. This is particularly bothersome because the cause might be the beta2 status of IE8 and not entirely a matter of a page's non-standardness.

This post has more confirmable-experience dimensions than revealed by the immediate content:

It is a prospect for confirmation of standards-compliant presentation via IE8 beta 2.
It is my first use of Blogger Labels (Categories in other blogging systems) to archive posts by categories as well as having weekly chronological archive pages and the buckets of individual posts made in the same month.
I am using Windows Live Writer image-uploading capability to FTP images to a directory of the blog and this is the maiden exercise to confirm that is functioning properly.

All of those arrangements are being explored with this single post. There's a principle about making only one change at a time that I am neglecting. OK, I feel lucky. This is the Blunder Dome, after all.

Update 2008-08-30T16:43Z I made some awful errors ("two-days" instead of "todays") and chose to tweak the page. I also notice that there are more problems that I don't know what to do with. It is time to come up with an IE8 mitigation approach. Stay tuned.

Update 2008-08-30T18:24Z I did achieve the three tangential objectives. As a result of that and the experience reworking the orcmid.com home page, I am adding an IE8.0 mitigation category for this post and others to follow on this and other blogs of mine.

Update 2008-08-31T02:46Z After developing further analysis and checking out further resources from Microsoft, I realized that I misunderstood the significance of the Compatibility View and what has the button be present. This page has been touched up accordingly.

nfoWorks: IS 29500 (OOXML) Moving Ahead?

2008-08-15T09:18:00.001-07:00

Technorati Tags: OOXML, IS 29500, ISO/IEC JTC1 SC34, ODF-OOXML Harmonization, OpenDocument Format, OASIS ODF TC

OOXML Struggling Over the Last ISO Hurdle?

Thanks to early word from Jerry Fishenden in the UK, we learn of today's press release in Geneva: The appeals to acceptance of DIS 29500 (Office Open XML Formats) are not sustained and preparation for publication of IS 29500 will proceed. There is one caveat, and that is the prospect of an appeal over the appeal resolution.

Meanwhile, DIS 29500 is now listed as deleted on the ISO site and we continue to wait for release of the IS 29500 documents to the public.

It is difficult to be excited about any of this absent availability of a specification to dig into. The maintenance process needs to get going and it would be really great if this situation were completely resolved, with publication of the edited IS 29500 specification, before the late-September ISO/IEC JTC1 SC34 meeting in Korea. The wheels of JTC1 turn slowly, with the best avenue for public input being the ECMA TC45 committee, where the gears also turn slowly and mostly invisibly. There needs to be greater transparency and public engagement, something that the federation system that funnels down into JTC1 SC34 does not support.

ODF as the Rôle Model

The situation for ISO/IEC IS 26300 (Open Document Format 1.0) is not particularly different, simply less encumbered at this point. Maintenance is at the OASIS Office Document TC. It seems unlikely that substantial work on ODF specifications will move elsewhere, given that this is where actual work is being done, unencumbered by multi-national hierarchic process and governance structures.

There are significant differences in transparency of OASIS maintenance in contrast with ECMA's record so far. There is a publicly-usable and visible comment list for submissions and public archives of the committee list and working documents, including a wiki that provides access to all proposals for the under-development ODF 1.2 specification. There is also an open discussion list for the proposal of an ODF Interoperability and Conformance TC. The public notice and review of that TC's chartering is expected soon.

Having joined OASIS as an individual, I expect to gain some very useful practice in anticipation of similar provisions for OOXML. In many ways, the division of my attention is going to be governed by where there is the least friction in the way of open discussion, non-duplication of effort (through knowing what comments there already are and knowing the views of others), and ability to review and make proposals. It will be unfortunate if the only open-to-the-public forum is a shadow created outside of any official OOXML maintenance bodies.

Cybersmith: Resources for .NET Interop

2008-07-13T11:33:00.001-07:00

Technorati Tags: interoperability, .NET, interop, Interoperability Forums, COM interop, .NET interop, Windows interop

Exploiting Interoperability Opportunities

There are a variety of techniques for integration and interoperability between execution models (POSIX, Win32, Java, .NET, etc.) and their platforms (via linking, inter-process communication, remoting, Web Services, Ajax, etc.). Interoperability arrangements can be extremely valuable in the reuse and repurposing of existing software and systems without requiring expensive, time-consuming, and just-plain-risky re-implementation.

The well-prepared cybersmith will want to avoid fragile, over-complex approaches and deliver something that is clear-cut, durable, and sufficient to the task. Seasoned cybersmiths will also anticipate future prospects for interoperability in their architectural approaches. That starts when thinking critically about immediate interoperability benefits in the software-development lifecycle: fitting into test harnesses, offering interface contracts, and re-integrating in cycles of build, integration, system test, and deployment. Anticipating interoperability between versions of the same system and its data is master-class cybersmithing.

Interoperability is a complex situation and not for the faint-of-heart. Unfortunately, it usually comes up under rush conditions. When I notice resources that may ease the shock and assist in realistic assessment of what's required for a particular kind of interoperability, I'll start a list like this first one.

.NET Interop Resources

Here's an alphabetical collection of resources for .NET interoperability that I originally compiled for an Interoperability Forum. That's a better place to submit additions than in the comments on this post, although I will notice them either way. This list will be updated from time to time. Eventually, it will link to a web location where more depth can be provided.

All About Interop: Connecting .NET to just about anything else. (web log)
The blog of Microsoft's Dino Chiesa covers general interoperability involving .NET. Although the blog tends to feature interoperability via communication arrangements, such as Web Services, there are discussions of topics related to linking and run-time interoperability. Click the "Interoperability" tag on any page to zero in on interoperability-focused posts. (Thanks to Enzo De Lorenzi.)
Tim Mallalieu and Jeromy Carriere: Enterprise Interoperability: .NET and J2EE (via Warren DuBois). article, Building Distributed Applications, Enterprise Architecture Patterns & Practices, MSDN Online, January 2004.
Although slightly dated, this article provides an overview and many links for those looking to address the different opportunities and trade-offs.
Gotchas with .NET (web article, via Silvain Trudel)
Jeff Cope addresses some specific .NET COM Interop considerations in the context of integration with PDMWorks Enterprise. Although some good understanding of Interop is required to apply these tips in other situations, they may be useful to review.
Jesús Rodríguez: Interoperability with Message Exchange Patterns Created Using BEA Weblog 8.1.3 (via Messages and Services). article, Building Distributed Applications, Enterprise Architecture Patterns & Practices, MSDN Online, January 2005.
Although grounded in a very specific case, the narration of the various considerations may be useful in considering other cases of web services interoperability. There are additional cases listed at the Messages and Services page. Here the .NET operation is as a client or server process of a web-service exchange.
Interoperating with Unmanaged Code. online article, .NET Framework Developers Guide, MSDN Online (undated).
An overview from the .NET perspective, dealing with pre-C++/CLI cases. This additional view may help form an understanding of what is involved. This assumes some knowledge of unmanaged code technologies (Win32, COM, ActiveX, etc.)
Thottam R. Sriram: Introduction to COM Interop. article, CLR Inside Out column, MSDN Magazine, January 2007.
This introduction takes a leisurely approach through the basics of interoperability between unmanaged COM code and managed .NET code. In addition to demonstration of all cases, there is discussion of debugging, with additional references for further resources.
Managed, Native, and COM Interop Team on CodePlex (project web folder, via Darren Stokes)
The main project of this effort is the P/Invoke Interop Assistant that can be used to see how C/C++ calls are translated to .NET calls, and how .NET entries can be accessed from C/C++ calls. There are help files and guidance on how parameters are marshalled between the Win32 native and the .NET environments. This is an useful way to understand Interop from execution environments that have C/C++ native calling and entry-point provisions.
Yi Zhang and Xiaoying Guo: Marshaling between Managed and Unmanaged Code. article, CLR Inside Out column, MSDN Magazine, January 2008.
This article provides an extensive analysis of the ways that calls and datatypes are in either direction between managed and unmanged code. An early version of the P/Invoke Interop Assistant is provided along with the downloadable examples.
MSDN Magazine (online issues).
The on-line issues extend back to March, 2000. The magazine articles can be found by issue, by column, and by topic (all in the sub-menu at the top of the home page). All code can be downloaded and individual issues can be downloaded in HTML Help (.chm) format.
MSDN Online (web site).
The on-line vresion of the Microsoft Developers Network provides extensive materials including the online issues of MSDN Magazine. The recently-updated site design provides easy search for interop-related materials, with opportunity for browsing and narrowing the search to more-specific material of interest.
Native and .NET Interoperability. (online article) MSDN Library.
This MSDN section deals with the ways that .NET and Native code can be mixed using the C++/CLI extensions of VC++ 2005 and later. It may shed more light on efforts to mix code as part of an integration across different development and runtime environments.
Adam Nathan: .NET and COM: The Complete Interoperability Guide. Sams Publishing (Indianapolis, IN: 2002), ISBN 0-672-32170-X pbk (2 volumes).
Although this book was published at the time that .NET 1.0 shipped, it is probably the most comprehensive treatment of the topic that will ever be found, with substantial code examples and description of the various tools that are useful in a great variety of combinations. Although the C++/CLI extensions were not available, and .NET is now richer, there are still topics here that are found nowhere else. This resource is also indicated if one is considering using COM as a way to access .NET from other native-programming environments. I also find it valuable in describing how to design for interoperability at either starting point for an interoperability solution.
.NET Gotchas (book, via Jeff Cope)
This 2005 O'Reilly book by Venkat Subramaniam lists a number of elements to watch out for in using COM Interop and also in deploying .NET Assemblies that use COM objects underneath.
Stephen R. G. Fraser: Pro Visual C++/CLI and the .NET 2.0 Platform. Apress (Berkeley, CA: 2006), ISBN 1-59059-640-4.
This is a thick and somewhat murky book. It tends to look at VC++ entirely from the C++/CLI language that the /clr option switches to. It does not connect the dots to native C++ that well. The pacing is leisurely enough that patient testing of the material by creating and varying the examples and samples may be useful. There are only two chapters focused on the interactions with unmanaged code.
The ServerSide Interoperability Blog. (web log)
Although silent since December 2007, this blog is directed to cross-platform interoperability. The archives have useful content. The sponsoring site TheServerSide.com tends to look at interoperability and integration from the Java side. Sponsoring site TheServerSide.net starts on the .NET side, with material on "language interop." These sites provide registration for e-mail notices of information and announcements, including ones related to interoperability.
VST.NET (via Marc's Blog Cabin). open-source project folder, CodePlex (LGPL 2.1 license).
This is an early version (0.2 as of 2008-06-21) of a developing worked example of a framework for starting from the C Language API that VST modules must offer, down through implementation of the module via .NET (exploiting Visual C++/CLI for the bridge, C# for the managed-code components called from the bridge). Virtual Studio Technology (VST) is an audio plug-in model defined by Steinberg Media Technologies and used to add audio processors and instruments to popular audio creation software. VST.NET is a real-world example of taking a specialized interface and allowing creation of VSTs using .NET via interop through C++/CLI. VST is a de facto standard for interoperable plug-ins of these fixtures. There is additional information on author Marc ~~Shapiro's~~ Jacobi's blog. The framework model may be more than what's needed in a particular interop situation. The modularization and use of a managed-code loader are instructive either way.

On the Interoperability Forums and the MSDN Forums (new ones and still old ones), requests for assistance with forms of .NET interop tend to arrive in many different places. Putting this list on the Interoperability Forums was intended to assist those who landed there looking for guidance. That is generally not the place where direct support is available, so I am reluctant to refer other requests there. This post is here as an alternative that I am more comfortable with as a known place that I can link to in responding to requests for .NET interop information in other places. When I find specific resources in applicable forums and web sites, I will lead to those from here too.

[update 2008-08-05T14:55Z Corrected Marc Jacobi's name as the author of VST.NET. I am correcting this on the Interoperability Forum also.]

A Conceptual Architect Am I?

2008-06-11T22:54:00.001-07:00

Technorati Tags: conceptual architecture, systems architecture, architectural diagrams

I am working on an article, "The Working of Computation," that applies What Computers Know and the secretly-connected Reality is the Model. This is a lead-up to the long-delayed What Programmers (Should) Know and, ultimately, Bridging the Gap that Cannot Close. This is seriously intended to have practical significance, no matter how abstractly it is couched.

In working on some diagrams, I noticed that some particular ones are very conceptual. I want to call them some sort of architecture, not some sort of model. I have used "performance architecture" as the term for a long time, and I was wondering whether "conceptual architecture" might be preferable.

With that in the back of my mind, I stumbled over this article earlier today: "CaaS Conceptual Architecture." Since the term seems to be in use, I wondered whether it is being applied in a way that works for my situation as well. Going from the cartoon, it appears to be a typical architectural "stack" diagram:

The Channel 9 videocast explains that this is about the pieces that can be combined to provide the architected facility (an instance of Communication as a Service, in this case). The color codes indicate the degree of control that is available for composing such a structure: red (little or no control), yellow (some control), blue (have a ball).

I'm satisfied that I am offering something similarly conceptual but from a different perspective and using a different form of diagram.

Just to check on the use of the term, I also did a search on "conceptual architecture." Here the trouble started.

On Wikipedia, I learn that "conceptual architecture" is a particular architectural style in the architecture of physical structures. Conceptual architecture features "the construction of a concept" in contrast to architectural focus on craft and construction. I would not have been surprised to see Christopher Alexander connected in some way, but that is apparently not the case. Well, as applied to computational systems, maybe I am constructing concepts, although I'm not familiar with those famous architects nor conceptualism as a doctrine. The doctrine has to with considering that universals have no objective reality and are, more or less, mental: objects of thought. I'm not prepared to make any metaphysical claims on that topic.

I recently had to satisfy myself once again that I am not a Platonist, and now I have do deal with being a Conceptualist or not?

I think I will ignore the implications and stick with diagrams being used to establish an abstracted, conceptual perspective of a subject and leave it at that for now. Some of the diagrams will be highly conceptual (a.k.a, diagrammatic) and also helpful in organizing a conceptualization of the subject. Ahh, I think I should just go finish the article I have in mind.

Cybersmith: Attributions for Code You Use

2008-06-06T11:31:00.001-07:00

Technorati Tags: Facebook, open-source licensing, attribution of software dependencies, software development, trustworthiness, cybersmith

If you choose to distribute open-source code, or create software that relies on licensed code of others, you'll have to deal with two issues:

what licenses you are willing to agree to for code that you rely on, and how you provide attribution to the third-party works that your software depends on
What license you will offer to others and how you want others to be able to understand your license requirements

These are interdependent considerations. Here I focus mainly on the first case because an useful example has come up.

Facebook has open-sourced a big chunk of its platform using CPAL, the Common Public Attribution License. This is an OSI-approved open-source license. Mike Gunderloy has opined that the attribution requirement that Facebook has specified in the license is onerous and designed to prevent others using the code because it requires that

"each time an Executable and Source Code or a Larger Work is launched or run, a prominent display of the Original Developer's Attribution Notice (as defined below) must occur on the graphic user interface (which may include display on a splash screen)"

where the specific attribution to be carried into derivative code is

"Attribution Copyright Notice: Copyright © 2006-2008 Facebook, Inc.
Attribution Phrase (not exceeding 10 words): Based on Facebook Open Platform
Attribution URL: http://developers.facebook.com/fbopen
Graphic Image as provided in the Covered Code: http://developers.facebook.com/fbopen/image/logo.png"

quoting from the Facebook version of the CPAL license. Gunderloy makes the observation that

"While this seems reasonable to recognize the work of the Facebook developers, it does act as a sort of 'poison pill' to prevent others from simply cloning Facebook on to their own sites - at least, others who don't want to give prominent credit to a rival."

Gunderloy seems to forget that one can always negotiate a different license with Facebook. He does point out that Facebook did not make use of the CPAL dual-license provision and the license as used is GPL-incompatible because CPAL is a derivative of the GPL-incompatible Mozilla license.

Dare Obasanjo has posted his "Thoughts on Facebook's usage of the CPAL as a 'Poison Pill' and Other Such Nonsense." Obasanjo does not think the attribution requirement is all that onerous, and I agree, although I think the requirement of prominent advertising is a bit heavy-handed.

I would not use CPAL-licensed code simply because the license is too bloody long and complicated. It also appears to be a reciprocal license, although I am not going to dive in close enough to be certain. I pretty-consistently avoid making derivative works of reciprocally-licensed code although I can imagine conditions where I would be willing.

One of my principle concerns in choosing an open-source license is that the license be dirt simple. I want recipients to easily determine and be very clear on what they are permitted to do, with simple conditions on compliance. I prefer something that can be fronted by a statement as simple as a Creative Commons Attribution Deed.

Obasanjo notes that the BSD license requires attribution too (you must carry forward a copy of the original notice). This has not bothered open-source efforts that use BSD code, even if under some GPL version. Sun pretty much has the practice down pat in its open-source efforts and closed-source ones too, with the THIRDPARTYLICENSEREADME that is commonly found in directories of programs such as OpenOffice.org.

When the extreme anarcho-libertarian wing of the open-source folk overcome their fear of being taken advantage of , they might notice another very important reason to provide accurate attribution. It provides an account of the provenance of their code and can be important in determining whether it might carry a later-discovered security exposure or bug reported in the original version. Attributions are an important feature of accountability, which is why I provide attributions whether or not they are required by licensed code that I incorporate or make derivatives of.

I am not interested in going overboard on the "prominent display" aspect. I think provision in conjunction with a Help | About ... menu item is fine. That's good enough for my attribution concerns. And, meanwhile, the standard rules apply: if you don't like the license, don't use the licensed work in ways where the unpleasant terms come to bear.

If you are wary of signing your own work and of providing attributions of your sources, that's fine. There is apparently a large community of like-minded folk who fear some sort of liability around simply being accountable. I will decline to let your code anywhere near any effort of mine, however. I'm also that way if the license for your code is not one I am willing to work under. It's simple, really.

I would normally have made this a simple comment on Obasanjo's blog, except some sort of Web 2.0-ish updates to the blog has my commenting-effort fail. There are some annoying pop-ups that I don't understand but that I can't dismiss, even though the site remembers my profile information and allows me to enter a comment. I just can't submit it because there is some sort of coComment failure every time I try. I'll have to hunt down Dare's e-mail address and let him know that this is a mess. Meanwhile, I've fleshed this out as an appropriate cybersmith topic.

DMware: ODMA's Dark Matter

2008-05-26T10:56:00.001-07:00

Technorati Tags: cybersmith, DMware, ODMA, interoperability, legacy software, software dark matter

I'd originally thought of posting about "ODMA in the Cloud." That doesn't work in the Web x.0 sense of the Internet "cloud." It does work to think of ODMA being "out there," unseen, yet its mass -- however tenuous -- is felt. Here's what I mean by all of that.

Into the Hidden Universe and Beyond ...

The Open Document Management API consists of three important elements:

A standard: the specification of an application interface and integration model that is relied on by a community
A Software Development Kit: an SDK providing header files and some samples that are intended to support programming against the API and developing services that offer the API (via the integration model)
Middleware: The ODMA Connection Manager, a software module that is installed as a bridge between applications (clients) and document-management subsystems (integrations) on the same computers

ODMA is a middleware arrangement in much the same way that ODBC and TWAIN are.

From the beginning with ODMA 1.0 in 1995, the elements of ODMA have been freely available: the API is free to use, and the middleware software is freely distributable and installable on Windows-equipped computers. (There have been occasional mumblings about support for non-Windows platforms, but that has not happened so far.)

In this way, ODMA has been cast upon the waters. There is no reason to know where ODMA integrations are installed, how many of them there are, and how varied they are. It can be presumed that ODMA adoption and usage has been steadily decreasing since completion of ODMA 2.0 in 1998, yet there is regular access to the ODMA Interoperability Exchange site where all of the elements remain available.

ODMA No-see-ums

In the course of active ODMA definition work, there have been three versions of ODMA:

ODMA 1.0, the original form
ODMA 1.5, introducing some additional administrative support and a form of federated search, and
ODMA 2.0, apparently over-reaching the sweet spot along with tidying-up some operability cases.

There are different specifications, ODMA Connection Manager implementations, and SDKs for the three versions.

The promise is that versions are upward compatible. In addition, an ODMA-aware desktop application can specify the version of ODMA that it is designed for. Configurations of ODMA are expected to decline to serve an application that depends on a version later than the level supported by the installed Connection Manager and document management system that is being accessed.

What we don't, and pretty much cannot, know is which

versions of ODMA Connection Manager are still in use, and in what quantity
versions of ODMA integration are supported by ODMA-compliant document-management systems
versions of ODMA are required by ODMA-aware desktop applications and how prevalent and varied they are, even though it is clear that ODMA 1.0 is the most-commonly required support that application programs require

Although there is a registry of ODMA-aware applications and ODMA-compliant DMS integrations, contribution of this information (and avoiding of duplicate DMS IDs and application IDs) is completely voluntary. Much of the information in the registry is out-of-date (as is much of the identified software) and there's not much to be done about it.

There is, in typical fashion, no good way to know how many implementations deviate from the specifications and how many may have "forked" the API and/or the Connection Manager software to accomplish a custom arrangement.

The only time that potential breakdowns of implementations and implementation-quality issues surface is when someone seeks support on one of the e-mail lists provided for ODMA support.

ODMA as Interoperability Laboratory

It is surprisingly commonplace that people implement specifications such as ODMA without consulting with each other or self-styled experts such as myself. It is clear that checking is mostly for successful operation with an important software product (typically Microsoft Word) or a given document management system (Novell Groupwise, Lotus Notes, and Documentum, among others).

There has not been, as well as I recall, any concerted interoperability testing of ODMA implementations beyond that required for a few early demonstrations at trade shows.

There are places where ODMA is known to be under-specified. These are invitations for developers to invent an answer. This is in addition to the ordinary risks of misinterpreting the specification and failing to confirm that interpretation (probably thought to be obviously correct).

Along with that, there are some promises made in the specifications that were claimed but not actively verified. One is the promise that the specification is upward compatible. For example, ODMA-aware applications developed against the ODMA 1.0 specification should be able to work perfectly well with other elements developed against the ODMA 2.0 specification. That promise has been broken, although it is not clear whether that has been much of a hindrance, and if not, why not.

It is both surprising and somewhat discouraging that there have never been any complaints about those breakages and a few others that have come to light over the years.

There are defects in the specification, there are defects in implementations, and ODMA integrations seem to be limping along mostly without serious incident, nonetheless. Of course, I am unlikely to be told where an ODMA integration was abandoned because it didn't work successfully with products that mattered in a particular situation.

I think that many features of the ODMA situation are typical of interoperability arrangements, how they can erode, and how they can be evolved toward a condition of reliability and stability. On the presumption that it is valuable to do so, at least for what it provides as useful experiment, I am going to recount a number of lessons that I have already experienced. These arose in my efforts to support software products having different programming and integration models (e.g., Java, .NET, and COM/ActiveX) than that of the native Win32 elements of ODMA.

Looking at how ODMA interoperability is enhanced is an opportunity to see how the same kinds of considerations also apply in more complex situations where demands for strong interoperability are expected to increase, not diminish.

Cybersmith: No-friction Bits and Pieces

2008-05-21T09:31:00.001-07:00

Technorati Tags: Cybersmith, Progressive Development, James Waletzky, Agility, Collaboration, Project Coordination, Microsoft OneNote

I'm a little distracted by James Waletzky's Tuesday, May 20 installment of Progressive Development: Motley says: "Spend less time in OneNote and more time in Visual Studio"

In this installment, Maven makes a paen to the ways that Microsoft Office OneNote is valuable in routine capture of notes and material related to a project. Maven also extolls use of shared OneNote files for collaborative creation and maintenance of all of those artifacts other than code that must be developed and maintained in the course of a software development. The idea is to reduce the friction in creation and sharing of such materials among the members of a development team.

I'm a big believer in the Extreme Programming "Travel Light" maxim (and my current career situation fits that perfectly). Although I have friends who swear by OneNote, and I do make use of it on my Tablet PC, I haven't caught the fever.

I'm concerned that the step-function for using OneNote (and the debt that accompanies yet-another siloed, proprietary document format) involves a large tax in terms of the level of Microsoft Office System commitment that must be made to support collaboration among members of a development team.

When the Office System (and Sharepoint) pill is too large (or the project is distributed in a way where it is simply impractical or unaffordable), other, lighter-weight approaches must serve. Project wikis come immediately to mind, as well as the use of distributed versioning for documents (to help in resolving conflicting updates and provide off-line as well as on-line working). Some people even use their bug-tracking system this way, although I don't find that satisfying myself.

What tools work for your (distributed) project-team collaborative-document and speedy note-taking and informal status needs? Do you use them for in-house projects, distributed projects, or both? How about collaborations in an open-source project? How do you navigate the need for sharing and collaborative documents and the desire for easy, rapid note-taking, project narration, feature discussions, easy time tracking, etc.

VC++ Novice: Is Native C++ a Dead Language?

2008-04-11T12:17:00.001-07:00

Technorati Tags: VC++ Novice, programming languages, software development, James Waletzky, cybersmith, C++ programming

This week, James Waletzky posted a valuable observation about the ongoing usability of C++: Motley says "Native C++ code development is obsolete." I recommend the entire post, the comments, and all of the other Progressive Development posts (with my cataloging here for an overview).

Here, I think is the key take-away and the main reason I am so keen to support VC++ novices:

Maven: ... I would argue that you are a better developer now because you had a C++ background. You understand what a pointer is don't you?
Motley: Don't insult me.
Maven: Do you ignore the concept of a pointer in managed code?
Motley: Absolutely not. There are times to pass objects by reference. There are times when using COM Interop that I have to worry about AddRef and Release. There are times when I do server-side development that I have to explore heap fragmentation issues (and in rare circumstances heap corruptions). Understanding a pointer is a necessity.

It's not just about pointers but storage structures, data representations, ~~efficient~~ economical use of resources, and ~~encoding~~ abstraction-manifestation techniques generally. It's also about having the choice of a tool that is fit for the job. ~~(and~~ These days, one can mix tools and interoperate with libraries built from ~~between the~~ different program codes, when it is worth the effort and maintenance complication. Learning to confine such complexity is part of the lesson.

It also comes down to what you want to become proficient at and how quickly, balanced by the importance of understanding the fundamentals deep enough to get out of trouble and also to avoid trouble in the first place.

For those who are concerned about Microsoft's continuing support for C++ development, the new version of MFC (the Microsoft Foundation Classes) and additional standard-library functions slated for the next version of the C++ standard have been released in a VC++ 2008 Feature Pack. The feature pack is not available for VC++ 2008 Express Edition, although there is expected to be some future availability (although that might only be the non-MFC additions as part of VS 2008 SP1 when available). Stay tuned.

[Update 2008-04-12T15:38Z: There was a phrase in one paragraph that nagged at me so I dove in and, in removing the nag, hacked up the paragraph fairly badly. I like the new one much better though. This is not unlike having a bit of code that just doesn't sit right, and sometimes it is the narrative in the comments that is the problem, some times it is the code itself.]

nfoWorks: What Are those Harmony Principles, Again?

2008-04-11T09:47:00.001-07:00

Technorati Tags: ODF-OOXML Harmonization, nfoWorks, Harmony Principles, open standards, document formats, interoperability

As part of readying the nfoWorks site for hoarding my hunter-gatherings for available materials in the wild, I have produced a refinement of the original attempt at some Harmony Principles. The latest attempt will now be kept part of the "About nfoWorks" page, and an early draft (0.1 beta) is now available.

There is additional information being filled in over the next several days:

Harmony Principles
Deliverables
Incremental Development
Start-Up Activities
Related Work
References and Resources

This is a typical first draft, with many sections simply being placeholders for content yet to come.

In arranging this material and the site, I did not have the foresight to create a blog or provide RSS feeds of the accounts being made on the site itself. This post is my penance for that. There is no way to make comments on nfoWorks, but you can always comment here and I will take notice.

Meanwhile, here are the basic ideas:

nfoWorks explores just how well documents can be made fully exchangeable when using a mix of different OpenDocument and Office Open XML implementations.
The first question: What prerequisites and restraints must be satisfied to ensure that documents are fully exchangeable and users can be confident that is the case?
The next question: Is there enough harmonization for users to willingly create, collaborate, and preserve their work using only only harmonious features of documents?

I offer these conditions of satisfaction for a high-level sense of what's intended:

Users can be confident their creations honor the format standards and depend only on their harmonious features. The Harmony Principles are honored by default.
Users can be confident that documents confined to a particular profile of harmonious features can be interchanged and interoperated with via any software programs that honor the Harmony Principles for the class.
In the event that a document relies on features beyond the harmonious level supported by a software product, profile-allowed limitation to supported features is explicit, automatic, and user-understandable.

Additional explanation and expanding detail are found via the About nfoWorks page.

nfoWorks: Tracking OOXML DIS 29500 into the Blue

2008-03-15T15:43:00.001-07:00

Technorati Tags: nfoWorks, DIS 29500, OOXML, ECMA-376, INCITS, standards-organization governance

I am not tracking every little move that happens in March 2008 as we determine whether or not the edited-per-the-BRM DIS 29500 will advance to an ISO/IEC IS 29500 and promulgation as an international standard. I have other work to do and other things to blog about. I shall return to my normal erratic mingling of topics. I promise.

What's It All About?
The US Score So Far
"Who Are Those Guys?"
Yes, but WHO Are Those Guys?
4.1 Not Voting
4.2 Abstention
4.3 The Loyal Opposition
4.4 Friends Across the Aisle
Eagerly-Anticipated Maintenance?

1. What's It All About?

I'm posting this article because of two interesting factors:

OOXML Maintenance might arise at ISO/IEC JTC1 Either Way

There is some interest in the US National Body for immediately creating a new JTC1 SC34 work item using an accelerated timeframe. The new project would produce an OOXML that has the BRM-recommended changes and also resolve more deliberatively all other BRM technical comments and more. This would be a new OOXML specification (revising IS 29500 if there is one of those).

The projected work is a little different depending on whether the edited DIS 29500 is approved or not. There's a suggestion that it could take 9 months to two years, depending on the situation and the quality of the edited DIS 29500. ~~I have not found the relevant JTC1 procedures to understand what these accelerated timeframe processes are.~~ I ~~also~~ have no statistics on the degree of under-estimation that happens with "accelerated timeframe" projects, although there is apparently strong oversight (section 5).

I raise this here as something to keep our eye on.   We have the prospect of a moving target for a few more years. The legacy solution may require legacy handling.
What's the US National Body?

There is some confusion about who these folks are and the interests they represent. Here's a scorecard for those playing along at home.

2. The US Score So Far

United States Votes to Approve Open XML. Jason Matusow reports that the US National Body has concluded a letter ballot that basically affirms its original approval of DIS 29500 in the September, 2007, ISO/IEC JTC1 balloting. This is the official result. There will be a resolution meeting for this, but I don't think there is any expectation that the outcome will change.

As Matusow points out, the vote for approval is 11-4-1 (yes-no-abstain) with one "not yet voted" (perhaps like voting "present"?). Either way, the 11 affirmative votes ensure a 2/3 majority.

Whatever adjustments are made, this is the US National Body Executive Board determination.

This is different than the V1 technical committee recommendation. This is the response to a letter ballot that was conducted following the V1 technical committee recommendation. This is the Executive Board determination.
The V1 technical committee recommended approval. There was a lengthy conference call on Friday March 7, 2008, where the BRM outcome was discussed and the committee then voted to recommend to the Executive Board that the original approval of DIS 29500 be affirmed.
The US Delegation to the Ballot Resolution Meeting voted "no" on some resolutions there. This was a small delegation led by Frank Farance. These were votes on editing of DIS 29500 and not on DIS 29500's approval or rejection at ISO/IEC JTC1. The most famous delegation "no" vote was a blanket vote on over 800 proposed responses (and although selected line-item voting was allowed, the US did not do that).

3. "Who Are Those Guys?"

In the United States, the promulgation of standards is accomplished by non-governmental bodies. There is a complex of such organizations. Keeping track of them is further complicated by the different ways they are associated with international activities, such as ISO/IEC JTC1 and other technical committees of ISO.

The pinnacle US standards organization is ANSI, the American National Standards Institute. ANSI accredits other organizations to develop standards. There are over 200 of those. Some of them, such as Underwriters Laboratory, are audited designators, but most standards developers don't certify processes or products. My favorite, after tallying the current list, is the Hydrogen Executive Leadership Panel. (By the way, OASIS is not one of these, nor are the W3C and IETF. Ecma International is not a U.S. organization, of course.)

INCITS, the InterNational Committee for Information Technology Standards, "is the primary U.S. focus of standardization in the field of Information and Communications Technologies (ICT), encompassing storage, processing, transfer, display, management, organization, and retrieval of information. As such, INCITS also serves as ANSI's Technical Advisory Group (TAG) for ISO/IEC Joint Technical Committee 1." With respect to ISO/IEC JTC1, INCITS is the US/TAG and acts as the US National Body in that capacity. It is the INCITS Executive Board that just held the letter ballot that affirms the US approval of DIS 29500.

INCITS has a pot full of technical committees (and they may have their own subcommittees and working groups of various kinds). Technical Committee V1 is on Office and Publishing Systems. They are a Technical Advisory Group for ISO/IEC JTC1 SC34. We see similar mirroring in other National Bodies. The German DIN NIA 34 has a similar function. V1 was once a tiny little group. Now there are 22 members and much is being said about that. Whatever the composition signifies, V1 has a broader charge than just dealing with DIS 29500:

"Since the scope of V1 is quite broad, it is necessary to create closer liaisons with other technical committees and special groups in order to establish good relations to avoid duplication of efforts. A number of new or extended areas are gaining interest in V1 such as Voice Messaging, Icons, Data Architecture, Font Services, MHS/X.400, Publishing Applications, Models and others related to Office and Publishing Systems."

If you wonder where INCITS came from and how it operates, here is more context:

"The InterNational Committee for Information Technology Standards (INCITS) is the forum of choice for information technology developers, producers and users for the creation and maintenance of formal de jure IT standards. INCITS is accredited by, and operates under rules approved by, the American National Standards Institute (ANSI). These rules are designed to ensure that voluntary standards are developed by the consensus of directly and materially affected interests.

"INCITS is sponsored by the Information Technology Industry Council (ITI), a trade association representing the leading U.S. providers of information technology products and services. ITI members employ more than one million people in the United States and in 2000, their revenues exceeded $668 billion worldwide.

"INCITS was founded as Accredited Standards Committee X3 in 1961. The last accreditation was April 19, 2001 under the name INCITS."

When I was last directly involved in these activities, around 1970, it was still X3 and the technical committees had designations like X3.4.2, X3J3 and X3V1 (if there had been a V1 then). At that time, ITI was known as CBEMA, the Computers and Business Equipment Manufacturers Association.

In addition to standards development being carried out by non-governmental bodies in the United States, the US also operates under a voluntary standards regime. For the most part, there is no mandating of standards and no one, not even participants in standards development, are obligated to adhere to a standard just because it exists.

Of course, adherence to specific standards can be required in procurement activities, and requirement of particular certification of standards-conformance might also be required. This is all pretty-much outside the purview of the standards developers in the voluntary model. (Some standards are given regulatory force, incorporated in building, electrical, food-processing, and other codes, but this is not under the authority of standards developers as standards developers in the ANSI regime.)

Here's the reminder that appeared at the bottom of the just-conducted letter ballot:

"Voluntary Standards are developed with the intention and expectation that the standards will be suitable for wide application. As their use is likewise voluntary, an affirmative vote does not commit an organization or group represented on the committee to the use of the voluntary standard under consideration. If you find that you cannot vote YES and wish to vote NO or ABSTAIN, please state this and explain the reasons for your position in the places provided."

4. Yes, But WHO Are Those Guys?

In January, there were 19 members of the INCITS Executive Board, but ~~at this time~~ there are only 17 voting members on this question. Here is how they turned out for the letter ballot.

4.1 Not Voting

AIM Global and Qualcomm, Inc. did not return ballots and might not (yet) be (voting) members ~~any longer~~.

Intel returned a "not yet voted" ballot, as already reported.

4.2 Abstention

The IEEE abstained. The IEEE, in addition to being a professional association, is also an accredited standards developer.

4.3 The Loyal Opposition

The four "No" votes are from Adobe, Farance, IBM (PDF text), and Oracle.

Of these, Farance is not a household word. Farance, Incorporated, is a technology firm that is heavily engaged in the development of standards, with some current emphasis on metadata and learning technologies. Frank Farance was the Head of Delegation to the DIS 29500 Ballot Resolution Meeting. As part of the August, 2007, deliberations on a US response to the DIS 29500 ballot, Farance cast this ballot recommending conditional approval (that is, "No, with comments").

A common theme is the impossibility of conducting adequate review of a 6,000 page specification under the fast-track procedure. IBM and Oracle both refer to "procedural irregularities" with IBM recording an extensive challenge to the validity of the INCITS process.

The Farance objection asserts that "A position of 'approval' is completely unacceptable to us" so it is a procedural appeal too, I suppose. The objection goes on to propose a remedy after enumerating the deficiencies. More about that below (section 5).

4.4 Friends Across the Aisle

Some of the "Yes" votes are from well-known organizations: Apple, Hewlett Packard, Lexmark International, Microsoft, and Sony Electronics. These five are not enough to carry the day.

The remaining votes include three substantial organizations that are mostly unfamiliar to me:

Electronic Industries Alliance (the EIA). This is a major industry association and also an accredited standards developer (think EIA RS-232 and RS-449).
EMC, self-identified as "the world leader in information management and storage." EMC provides solutions for mainframe, IBM DB2, Microsoft, Oracle, and SAP. Documentum is now their product. Oh. I need to get out more.
GS1 US, another non-profit accredited standards developer that focuses on global supply-chain solutions. If you need a Universal Product Code to stick on your product packages, you get it from GS1 US.

The last three "yes" votes are of an entirely different breed:

U.S. Department of Homeland Security, with an expression of alignment with NIST and DoD
U.S. Department of Defense, after consultation with other Federal agencies and expressing alignment with NIST
U.S. National Institute for Science and Technology (NIST). This is a Federal agency under the U.S. Department of Commerce. Formerly know as the National Bureau of Standards, NIST is not in charge of standardization in the United States. NIST/ITL (the Information Technology Laboratory) is an accredited standards developer, however. NIST has, in the past, drafted Federal Information Processing Standards (FIPS) that apply to agencies of the U.S. Federal Government. These are often existing public standards adapted for governmental use. There is an extensive report on NIST participation in INCITS with respect to ODF, OOXML, and the DIS 29500 process (PDF text). There is also a NIST press release on the matter.

5. Eagerly-Anticipated Maintenance?

I'm intrigued by the following statement in the NIST ballot:

"With respect to the 'Additional INCITS/V1 Recommendations Not Related to this Vote,' included in INCITS Letter Ballot 2558, NIST interprets these recommendations to be that the USNB should support one of the following options:

"1) Should DIS 29500 fail to achieve the criteria for approval as an ISO/IEC standard, DIS 29500, as modified by the BRM, should be submitted to JTC 1/SC 34 for fast processing, via the JTC 1 combined NP [New Work Item Proposal] and CD [Committee Draft] ballots process, and progressed under the JTC 1 accelerated time frame.

"2) Should DIS 29500, as modified by the BRM, achieve the criteria for approval as an ISO/IEC standard, to correct remaining issues from the BRM, SC 34 should approve an NP for a revision of IS 29500 using the JTC 1 5-stage process under the accelerated time frame."

No "Additional INCITS/V1 Recommendations" appear as part of the ballot form. There is something about them in the Farance ballot though:

"Although we disapprove of the DIS 29500 at its present state, Farance Inc. is supportive of the 29500 technologies becoming an ISO/IEC standard. We want the work done right and done quickly to address industry and international needs. Farance Inc. has proposed in INCITS/V1 two New Work Item Proposals [NPs] for SC34: one NP if DIS 29500 succeeds and a different NP if DIS 29500 fails. These NP proposals were acknowledged in the INCITS EB ballot wording. We hope these NPs will be a starting point to trying to fix/finish issues within the 29500 text, and we hope these immediate needs can be addressed over the next 9-14 months."

It is not clear that NIST is throwing its support behind that idea or is simply clarifying its understanding. ~~I have failed to find the particular JTC1 procedures that are referred to as "the JTC1 5-stage process under the accelerated time frame." I'm still looking.~~

[Update: There are three different schedule templates for JTC1 New Work Item Proposals: accelerated (24 months to published standard), default (36 months), and extended (48 months) on the lines of [1: section 6.3]:

Schedule	Committee Draft (CD)	Approval Draft (DIS)	Published Standard
Accelerated	none	6 months (enquiry draft)	24 months
Default	12 months	30 months	36 months
Extended	12 months	43 months	48 months

The entire section on Programme of Work is important for an understanding of the process [1: section 6].

The JTC1 5-stage procedure has 6 stages (0 to 5), [1: section 12.1]:

Preliminary: A study period is underway [probably brief for OOXML]
Proposal Stage: the NP is under consideration (requires balloting and formulation of the work, assurance of participation, and other important formalities, [could be brief for post-IS 29500, might be contentious otherwise])
Preparatory Stage: Working Draft (WD) under consideration [probably using either IS 29500 or updated ECMA-376 as initial WD]
Committee Stage: A Committee Draft is under consideration [something tells me there will need to be something like that either way, using normal processes]
Approval Stage: An FDIS is under consideration [more-or-less where the Fast-Track DIS 29500 process started and where DIS 29500 is right now]
Publication Stage: An IS is being prepared for publication [a process yet to be completed if DIS 29500 is approved]

My personal reality check suggests that the odds of this getting off on the right foot (starting work around September-October 2008) is if there is a published IS 29500 as a concrete starting point. Whatever process is put in place, there must be room for feedback from implementation experience and any working agreements that are formulated as part of document interoperability efforts. Putting the necessary technical muscle into an SC34 working group and coordinating with NB counterparts (such as INCITS/V1 and others) will be daunting and time-consuming. Some way of providing intermediate guidance and (provisional) interpretations may also be important. I'm not betting on the accelerated timeframe even under the best of conditions.]

Here's what I still think. OOXML is the new COBOL. Really. That's a good thing (with a 40-year life span, for starters), if the energy and pace can be achieved with the requisite quality and care.

[1] ISO/IEC JTC1: ISO/IEC JTC 1 Directives, 5th edition, version 3.0. JTC001-N-8557, 2007-04-05. 1696kB PDF download available for download at ISO/IEC JTC1 Public Information, Procedural Documentation.

[update 2008-03-17T19:45Z: I received a very helpful e-mail through the NIST Media Relations Office, pointing me to the relevant sections of the JTC1 directives. The selections, summaries, and interpretation are all mine.

update 2008-03-16T15:45Z: I learned from an Eric Lai Computerworld article that AIM Global and Qualcomm are apparently new members of the INCITS Executive Board. I updated my observation about that and who are voting members accordingly.]

nfoWorks: The ISO/IEC Harmonization Opportunity

2008-03-11T12:53:00.001-07:00

Technorati Tags: Rick Jelliffe, OOXML, ODF, ODF-OOXML Harmonization, DIS 29500, nfoWorks, Harmony Principles, Rob Weir, Jan van den Beld

What Is in the New Draft of OOXML? Rick Jelliffe has put up an excellent post on the history of the ODF and OOXML progressions and on the results of the Ballot Resolution Meeting in Geneva. The entire post is a valuable summary. With regard to the improvements of DIS 29500 (if approved by ISO/IEC JTC1), Rick's description of improvement is a valuable supplement to the offerings from Alex Brown and Jan van den Beld that I have applauded concerning BRM Closure in my just-updated "In Search of Initiative" post.

With regard to the prospects for harmonization, I found this Jelliffe tid-bit particularly interesting:

"Many other related issues were also discussed in the hallways at Geneva. For example, the German DIN standards body is preparing a cross-mapping list to match features in OOXML and ODF: there really is very little information on this currently, despite the confident assertions that ODF can/cannot handle everything that OOXML does and vice versa. The Italian standards body is seeking to work on conformance suites for testing: obviously the schemas and BNF grammars allow validation testing of instances for document conformance, so I presume the test suites will be more concerned with application conformance. ISO/IEC JTC1 SC34 has been making various preparations to establish an effective and responsive maintenance regime: ODF could also benefit from this effort."

I am intrigued by anything that happens in Italy. (Perspective: If I mention this to Vicki, she'll say, "More trips to Tuscany!" Around here, visiting Berlin would be wonderful, living in Italy would be heaven, and taking the bus to meet-ups in Redmond is not a consolation for her.)

More than that, I think this is what our attention should be on and I am happy to see that people are turning their interest to this matter.

Jelliffe also speaks to issues of maintenance. The maintenance prospects are important and I want to feature these passages (ripped from context but, I trust, preserving Jelliffe's sense of it), with my emphasis:

"If the new [DIS 29500] draft is adopted as a standard, it does not remain static but can be “maintained” by the relevant ISO/IEC JC1 committee, SC34, Document Processing and Description Languages. Procedures exist for National Bodies to submit Defect Reports, which again attract the Editor’s attention and National Body voting acceptance, so the kind of process seen at the BRM becomes an ongoing effort, if there is enough interest by National Bodies."
"The upshot is that, if DIS29500 mark II and ODF 1.2 both get accepted as standards, by the end of 2008 we should have two standards which together can thoroughly cover the field of representing current and legacy office documents, each representing one of the two dominant commercial traditions, with both under active and significantly open [PDF download] maintenance to fill in the remaining gaps and to repair pending broken parts, with clear cross-mapping to allow inter-conversion, with an increasing level of modularity so that the can share their component parts, and at least with a feasible agenda of co-evolution [PDF download] and other kinds of convergence.
"And if we play our cards well, both traditions will have significant competitive motivation to accommodate the technical requirements of their competitors. Viola, harmonization?"
"I think we did pretty well in the Australian delegation, in getting many of our issues addressed completely and most of our issues addressed in part, but (like any standard!) the more you look the more holes you see. There are so many improvements that can and should be made by pro-active maintenance."

It is the opportunity before us that I find heartening in Jelliffe's analysis. I have the same reaction to the Patrick Durasau analyses that Jelliffe links.

Contra Durasau, Part 1. Rob Weir has a rather different post today (promising even more in his choice of serial title). Weir is articulate and very bright. (I base this on the fact that he plays chess, shows knowledge of philately, and probably also writes software, all better than I do.) He lays down a challenge for ECMA TC-45, a challenge that I think requires some pro-active behavior to remedy (and the W3C model, if not the OASIS one, comes to mind for a participation-inviting alternative):

"Where are the minutes from Ecma TC45 teleconferences? Where are the public archives of their mailing list? Where is the list of individuals participating in the TC? Where is the list of voting members? Where are the public comments they have received on OOXML? You call this open?"

If I understand Jan van den Beld on this topic, it is pretty much up to TC45 to deal with this (and put in the effort required, although there is ample computer-based support these days).

Weir has a great deal more to say, and it is a worthy read. It bothers me that it is a worthy read, and I understand my concern better after coming across Jelliffe's appraisal. Here's what I see:

It is easy to look at past conduct of Microsoft, including how it operates the revision cycle of its productivity and developer software, and interpret that as unequivocal evidence for no possibility of any other outcome around interoperability and pro-active support for broad-based, open-process industry standards when Microsoft is at the table. (I don't know how the joint Microsoft-IBM work on web services survived this imperative, unless it is something like certain historical non-aggression pacts designed to cynically divide up the spoils.)

Weir's no-possibility argument is his basis for claiming that DIS 29500 should not be approved.

For me, to act from that position of no possibility is a guarantee of no opportunity. I stand for the possibility of broad interoperability arrangements. There is an opportunity here, and it is up to us to seize it. If attention drifts away and maintenance and interoperability suffer, that is a challenge for those of us who see this as worthy and important work. Of course it feels risky. There is always uncertainty in changing the world, even in these small ways.

I'm looking forward to the bringing into being of the world that Rick Jelliffe and Jan van den Beld see as possible.

nfoWorks: In Search of Initiative

2008-03-10T13:37:00.001-07:00

Technorati Tags: Ecma International, BRM, DIS 29500, ECMA-376, Document Interoperability Initiative, ODF-OOXML Harmonization, nfoWorks, Harmony Principles, Alex Brown, Jan van den Beld

I've been waiting to see more about the Microsoft Document Interoperability Initiative so that I can assess its value in the context of the Harmony Principles. I'm wondering what nfoWorks might provide by way of resources, and vice versa. The potential changes to ~~ECMA-379~~ ECMA-376 that will be part of an approved ISO/IEC DIS 29500 also need keeping an eye on. Here's the interim situation.

Unfolding Document Interoperability
Format Plug-and-Play
Waiting for BRM Closure

1. Unfolding Document Interoperability

The first actions under the Document Interoperability Initiative were announced in a March 6 press release [1]. The release reported that day's event in Cambridge, Massachusetts, with additional events to follow around the world. There was not much meat in the announcement. I found the approach to be slanted in an odd way:

"The Document Interoperability Initiative focuses on bringing vendors together to promote interoperability between document format implementations through testing and refining those implementations, creation of format implementation test suites, and the creation of templates designed for optimal interoperability between different formats."

This fits with having plug-fests to test interchange and also having round tables to learn the concerns of the vendors who ask to participate. It seems straightforward enough. Yet I find an odd disharmony in the statement from Jean Paoli,

"“Microsoft believes that the industry has a responsibility to come together to address the interests of users in achieving greater interoperability and effective data exchange between widely deployed document format implementations.”

The notion that putting vendors together addresses user needs occurs in the statements of other participants as well. Somehow, I thought we had reached the point where we know to engage users and representatives of significant user communities to find out what their interoperability needs are. I guess I think I know what users want too. Don't you? Out of fairness, commercial firms may have a good sense of what customers are telling them. I suspect we will see more about that as the Document Interoperability Initiative continues. Still, I think the notion of "industry" here is too narrow.

Meanwhile, the press release led Mike Gunderloy to question the focus on commercial vendors with closed-source products [2]:

"So far, though, participation in this new initiative seems to be limited. Perhaps Microsoft rushed to get this together to prove that they mean open business, but the only open source company involved in the first round of meetings is Novell. Other participants (DataViz, Mark Logic, Nuance and QuickOffice) all produce closed-source products that work with one or another open format.

"If the idea here is to prove that Microsoft gets along with and wants to support the entire document-based community, they're going to have to do better than that. Until fully open-source projects such as OpenOfice and NeoOffice sit down at the table, suspicions will remain that Microsoft continues to guide the interoperability testing for their own benefit."

It is difficult to say what went into the initial selection and what the invitation process might have been. Sometimes the most difficult part of having multiple vendors in the same room testing their wares is arriving at acceptable mutual NDA agreements. Even if there is no need to share or discuss their code, there is information that can be taken away concerning product directions, product readiness, and also defects exposed during any lab exercises. With regard to "fully" open-source projects, I suspect it depends on whether those projects are interested and organized to participate. I would hope so.

The Microsoft agent-in-charge is Interoperability Evangelist Craig Kitterman, whose blog has interesting photographs and event details. Kitterman ends his summary with an open invitation [3]:

"I want to call out here that this is open to any vendors who are implementing open standards based document formats and are interested in working with a broader set of folks with common goals. If you are interested in participating, please feel free to contact me directly (ckitter@microsoft.com) and we can work together to see how to integrate your company, product and ideas into the next session.

"If you cannot participate directly in one of the events but would like to add your comments as to what kinds of things we can and should do as a community in the interest of document interoperability, please feel free to comment or shoot me email."

We're still in vendor church, but now there's a person who stands up for the activity. Also, there are interesting notes on the round table held during the event.

I assume that the Microsoft-Novell joint laboratory provided the setting for the Massachusetts event [4]. Craig Kitterman's blog has a slide show that provides clues. Craig reports that the next events will be in Korea and Germany. I can't tell whether permanent facilities are available for periodic events and plug-fests, such as Port 25 and Building 20 in Redmond.

2. Format Plug-and-Play

The press release on the Document Interoperability Initiative also invokes the Interoperability Principles as part of announcing a new drop of the open-source OpenXML-ODF Translator software. The tie-in:

"Microsoft has committed to support future releases of the translator taking advantage of the improvements in Microsoft Office converter APIs announced as part of the interoperability principles on Feb. 21 to provide a better integrated experience for customers to open and save ODF files. These APIs and the guidance provided by the OpenXML-ODF Translator project will also make it easier for users to take advantage of other document formats, such as UOF and DAISY."

The OpenXML-ODF Translator project (well, that is what it is called) will eventually be a demonstration of the use of the new Microsoft Office interfaces for smooth integration of new formats.

[Note to self. At some point, it would be interesting to see if it ever makes sense that there be a separate conversion interface for back-and-forth with particular harmonization levels/profiles, even of OOXML format. I have no idea at this point.]

3. Waiting for BRM Closure

At the moment, it is not possible to do anything concrete and detailed with Office Open XML harmonization until it is known what the final ISO/IEC status of DIS 29500 becomes and what the final edited specification is.

For now, the only stable specification for working with Office Open XML is ECMA-376. There's plenty to deal with there. At the same time, I want to anticipate how an approved ISO/IEC DIS 29500 might ~~impose the need for some~~ introduce adjustments (with accompanying transition provisions). The March 29 closure of the DIS 29500 ballot process will certainly be soon enough to intercept anything that happens with nfoWorks and the Harmony Principles in the meantime.

I have obtained the information that Alan Brown has made public so far [5]. I've found useful guidance on how ZIP usage and Open Packaging Conventions (OPC) will be adjusted. I also searched out some items that address conformance and levels of application. There was a proposal to provide for application descriptions that may be a future opening for what I have been vaguely referring to as interchange profiles.

My attention was drawn to Alan Brown's material by hints in a post from Jan van den Beld [6]. That post ~~appears to have been withdrawn~~ was withdrawn and replaced by a stronger, more-complete version. ~~I am hopeful that~~ Van den Beld ~~will provide a replacement that~~ shares his valuable perspective on the history of Fast Track submissions.

[1] Microsoft: Microsoft Launches Document Interoperability Initiative. (press release) PressPass - Information for Journalists, microsoft.com, 2008-03-06 (updated 2008-03-10).: This release confirms that something is happening. I was left with many unanswered questions about the actual structure of the initiative and whether the labs were definite facilities or ephemeral events (at definite facilities).
The humor of the press release update is that an occurrence of "Microsoft Office Open XML" was replaced with "Office Open XML," reflecting the separation of OOXML from Microsoft's possession. And while one can't rewrite Jean Paoli's predilection for unqualified use of "Open XML," it is unfortunate that this off-putting contraction is used throughout the press release.
[2] Mike Gunderloy: Document Interop from Microsoft. (web log entry), OStatic, 2008-03-07.: I'm not sure ...
[3] Craig Kitterman: Document Interoperability Roundtables & Labs - Take 1: Cambridge, MA. Craig Kitterman's Interoperability Community Blog, msdn.com, 2008-03-07.: The slide-show and description of the event provide the most visibility on this activity so far.
[4] Tom Hanrahan: Microsoft and Novell Interoperability Lab Announcement. Blogs, Port25, technet.com, 2007-09-12.: Although focused on the server farm, Hanrahan mentions the areas of interoperability that this joint operation supports: virtualization, web-services management, and identity federation. In the November 2006 announcement of this activity, document-format compatibility was also part of the charter. That still seems to be the case.
I had bet myself that this lab would end up in San Jose or maybe Idaho. Instead, Microsoft and Novell found a more-interesting way to require comparable travel distance from Redmond and Provo. I keep forgetting that Novell is already in Cambridge, MA. Microsoft is expanding its own presence with a Microsoft Research operation (not to be confused with the one at the original Cambridge).
[5] Alex Brown: BRM Documents. There is no end, but addition (web log), 2008-03-06 (updated 2008-03-10).: The four downloads provide for ample reference until the post-BRM ballot closure period ends. The original ballot comments and the ECMA TC45 proposed responses are not included. We'll have to wait for all of that to be sorted out.
[6] Jan van den Beld: The BRM for DIS 29500: Open XML, a last-but-one, normal climax in FT! Jan van den Beld (web log), 2008-03-09. Replaced 2008-03-11 as "A new view of the BRM for DIS 29500 has emerged: Consensus prevailed.": This post has been deleted from the blog and from the blog's RSS feed. It's conceivable that van den Beld reconsidered how much information to provide about aspects of the Ballot Resolution Meeting. I don't know.
The value of this post to me is the great historical perspective on the ECMA-ISO relationship and how often the Fast Track (FT) process has been used with success. ~~I look forward to that portion being reposted in some form.~~ I am grateful to see that this useful and informative post has been re-issued with additional material.
This post brought my attention to some of the resolutions that might impact the nfoWorks effort. ~~Fortunately,~~ These are all ~~available~~ locatable in the Alex Brown materials; the original version of this post inspired me to look for them. Now you can too.

I received my third Waggener Engstrom e-mail last week. It pointed me to the Document Interoperability Initiative press release. There was nothing there to talk about, with all links being to materials already covered about the Microsoft Interoperability Principles. Later, it seemed that the OStatic folks could find some value in Craig Kitterman's more-substantial report and contact. Since I did not want to go through the pain of registering at yet one more site in order to comment at OStatic, I chose to make the introductions here.

The ultimate trigger for this post is the BRM information from Alex Brown and the (currently-unavailable) background from Jan van den Beld.

The nfoWorks site is bootstrapping along. The goal is to having enough structure in place that I can start compiling notes on available materials. My first interest is identifying what is needed to build up some document-processing infrastructure with progressive layers of abstraction for processing of open-standard document formats under the Harmony Principles. The first clue that the site is becoming something humans can use will be appearance of a genuine home page. This will have the first links to content other than site masonry and plumbing.

[update 2008-03-11T17:14Z: I don't know how long it will take me to stop missremembering the number of ECMA-376, especially since all I have to do is look down at my computer desktop and the folder that is named for the specification it contains. Since I needed to correct one designation at the top of this post, I added a link to the Ecma International source of the specifications and adjusted other statements in the body of this article. I also checked Jan van den Beld's blog and am happy to report that he has put up a refined version of the deleted post that I found so valuable.]

nfoWorks: The Harmony Get-Ready

2008-02-27T10:44:00.001-08:00

Technorati Tags: ODF-OOXML Harmonization, Harmony Principles, nfoWorks, Microsoft, Interoperability Principles, Document Interoperability Initiative

see also:

2008-04-11: nfoWorks: What Are those Harmony Principles, Again?
About nfoWorks and the nfoWorks Diary for additional updates

ODF-OOXML: nfoWorks for Harmony? On February 7, I distilled the Harmony Principles onto a web page. I was looking for a reaction and also giving myself a breather after finally declaring myself on the matter. I did begin setting up a web site and laying out a foundation on which to commence harmonization of open-document standards by limiting processor implementations to only harmonized content.

Microsoft Interoperability Principles Announcement. When I listened to the February 21 Interoperability Principles announcement and related initiatives, I made note of Bob Muglia's comments about a Document Interoperability Initiative. The import of that didn't sink in until I was finally completing my lengthy "Interoperability by Design" post bearing my impressions and analysis of the announcement.

Welcome, Document Interoperability Initiative!

here is what the February 21 Interoperability Principles declaration says will happen as one of the steps under Principle III on Data Portability:

"Microsoft believes that the industry has a responsibility to come together to address the interests of users in interoperability and effective data exchange between widely deployed document format implementations. To this end, it will launch a Document Interoperability Initiative that will include an ongoing series of labs around the world where implementers of document formats optimize data exchange between implementations, community development of conformance testing suites for popular formats, and publication of document templates that enable optimized interoperability between different formats."

During the announcement teleconference, Server and Tools Senior Vice President Bob Muglia provided this plain-language statement:

"We're also announcing a document interoperability initiative to ensure that the documents that are created by users are fully exchangeable, regardless of the tools that they are using."

I haven't discovered a web presence for this initiative (and the related domain names are available). Meanwhile, I will be keeping my eye on the Interoperability Forum, the Open Source Interoperability Initiative, and of course, the OpenXMLDeveloper.org and any developer-focused ODF sites I am able to find.

Whither Harmony Principles?

So, maybe I don't have to do anything here? I'm not sure. It may be that a lot less original work will be needed to have the harmony principles ~~are~~ realized in available document processing appliances. There is certainly important cross-fertilization with translation, conformance testing, and templating activities, especially those that are provided under open-source terms.

I have several reasons for continuing to develop nfoWorks at this point:

I want to illustrate implementation of document processing as part of nfoWare, and while digging into the processing of a full-up open-document format is interesting, that is heavier and more focused than the level required for nfoWare. nfoWorks is a better place for the research, gathering of materials, and development of detailed implementations that target the Harmony Principles.
The Document Interoperability Initiative will attract many interests and energies. Those that provide compatible open-source artifacts will be important to draw on in nfoWare as are artifacts of the related translator efforts. Likewise, any unique contributions that nfoWare surfaces will be available to the Document Interoperability Initiative, translator activities, and document-generation projects. We need to see more on all fronts.
The more overlap, the less work for me, I say. Still, the achievement of any strict implementations of Harmony Principles may not be the primary interest of others. I want to cling to that objective for the important purpose of finding out how much can be done. The goal is assuring users that they don't have to worry whether the documents they are creating are fully interchangeable among harmonizing implementations of the various open-document formats. Users will be able to count on the safe interchange of their documents.
Establishing document profiles for use by communities of (collaborating) information workers is an open problem. It might not even be well-defined. The Harmony Principles establish a rigid test, and we need to see how far one can go to honor that. It is also valuable to learn the conditions under which relaxation (or qualification) of the principles can be safely tolerated. Perhaps more important than that is figuring out how the creation and use of profiles that constrain documents can be done in a way that makes sense for the information workers whose labors will be supported and impacted by them.

That said, I am perfectly happy to give it all away. I will shepherd things along with an eye to exactly that outcome.

After defining nfoWorks, I realized that I should also register a domain name or three. I have accomplished that. Now I am bootstrapping up an nfoWorks.org site so I can start more notes, checklists, and a repository of prerequisite materials. At the moment, that site is as interesting as staring into an empty excavation waiting for a foundation to be framed and poured. As soon as the rain stops ...

[update 2008-02-28T16:59Z: I removed an extraneous work and sharpened up the first item on the connection between nfoWorks and nfoWare. I may long regret that I made the names so similar.]

DMware: Documents as Evidence

2008-02-20T11:10:00.001-08:00

Technorati Tags: DMware AIIM C22 Standards "Evidentiary Support" "Records Management"

Over the years, the AIIM International sponsorship of standards has included fostering of committee C22 on Evidentiary Support for Electronic Information. I just learned that C22, which is tied to ISO TC171/SC3, is being rebooted. I confess that I lost track of this work and I'm heartened to see announcement of its rejuvenation:

Why Comply with Standards
WHEN YOU CAN WRITE THEM?
When you join the AIIM C22 Evidentiary Support Standards Committee, you'll play a pivotal role in the creation of new standards for the legal admissibility of electronic documents in courts across the country.
Led by Co-Chairs Bob Williams and Paul Doyle, the committee deals with law and IT in relation to enterprise information management, with particular focus on information as evidence... for or against the enterprise.
About a quarter of our stakeholders are attorneys who are conversant with IT. The rest of us are mostly IT people who are conversant with legal issues, legal thinking, and the nature and practice of law. We are focused on enterprise business activities that are conducted electronically. Our committee is trying to stay even with - and, if possible, ahead of - the technology curve, in order to advise IT designers on strategies for making e-records trustworthy in the eyes of the law. We also work on strategies for enterprises trying to identify and dispose of unwanted electronic information.
We concentrate on evidentiary considerations for the design of e-business systems, on the retrieval of information in discovery, on the erasure of information that the enterprise does not want to hold, and on intellectual property considerations in the management of electronic information.
The work of AIIM C22 is substantive, intellectually challenging and its influence is wide-ranging. We invite you to join us.
Send your contact information to standards@aiim.org and copy C22 committee member Paul Doyle, at paul@proofspace.com so that we can add you to the C22 e-mail list. There you'll learn about upcoming meetings and ongoing projects in which you can get involved.

For further information and general interest in the focus of AIIM-supported standards activity, visit the AIIM Standards page. There is also an AIIM Standards Watch blog.

Although the evidentiary nature and use of electronic information, including documents and source code, is not the most exciting thing for Information Technology organizations to be concerned about, understanding the importance and the risks of how electronic information is managed is becoming increasingly important.

It occurs to me that the folks who object to the ISO Standardization of DIS 29500 (Office Open XML) would place a different spin on the invitation's headline. I think the spirit of this announcement is clear even if the headline is a little over the top.

I also think work to make the legal implications of IT policies and practices around electronic information more understandable is very important. It is one of those odd situations where it is desirable to make discovery (and auditing) more efficient because of the burdensome cost that it can involve, while also making sure that electronic information is destroyed at appropriate times to avoid the secondary burden of witch-hunting by discovery. C22 is a forum where knowledgeable people worry about that tension and how to delimit it.

[Cross posted on Professor von Clueless in the Blunder Dome and Orcmid's Lair because of overlapping communities of interest.]

VC++ Novice: DreamSpark for Students

2008-02-19T16:17:00.001-08:00

Technorati Tags: VC++ novice, cybersmith, student programming, computer enthusiasts, DreamSpark, self-directed learning

Bill Gates talks about Free Software, Students, and Technology

Yesterday, Bill Gates spoke about the DreamSpark program to put Microsoft's professional developer tools into the hands of students at no charge. There is an exciting list of available tools, including Visual Studio 2008 Professional and the complete Expression Studio.

The initiative is now available in eleven countries around the world. Within those countries, you must be a student at one of the recognized institutions or other recognized student organization. Your student status will be verified with the appropriate organization, and it must be reviewed every 12 months.

The program will rapidly expand to more educational institutions and countries. For now, students must be enrolled in a recognized post-secondary (after high-school) program. The program will be extended to high-school students at a later time.

DreamSpark software is downloaded directly by the students, rather having to be obtained through academic departments and campus book stores. This should make the packages more-consistently available.

There are some restrictions. I also have some concerns.

Restrictions

First, the software is only available for academic use. From the FAQ for Students:

"We’re giving you the software for non-commercial use to support and advance your learning and skills through technical design, technology, math, science and engineering activities."

Secondly, the software is only available to use so long as student status is maintained. From the FAQ for Administrators:

"This benefit is available to all tertiary (post-secondary) students around the world who are attending accredited schools or universities. However, this program requires all students to have their status verified by an authorized verification source."

Not being a verifiable student, I am unable to look at the complete license and explore the download process. You will need to find broadband access and be able to download and record CD-ROM and DVD-ROM images. You should also explore any restrictions on redistribution and on distribution of software that you produce.

Concerns

If the DreamSpark idea appeals to you and you are a qualified student, remember that these products are designed for professionals. There is no provision of tutorials or introductory materials for learning how to use the various programming languages and other resources. You will need to invest in separate books and resources beginning with these tools.

It may also be difficult to find a community of other students and beginners who are also learning to work with these tools.

I suspect that there will be more material and more community as time goes on.

If you are a beginner and are not taking courses or using books you already have for the fundamentals, I recommend that you look at one of the Express Editions and use that to calibrate whether you are prepared to take on the additional features and capabilities of the professional tools without considerable assistance. This is also a good place to start until you are prepared to declare yourself to be an advanced student.

I don't want to discourage you; this is a great offer. But you should qualify your expectations. Mastery of these tools is not something that happens overnight or even in a single university term. If you want to use these tools for self-directed learning separate from your course work, it is even more important for you to understand the investment of personal time that will be needed.

My recommendation: Go for Expression Studio and Windows Live. Use the Express Editions for everything else until you are prepared to step up to Visual Studio 2008 Professional.

What Others Are Saying

Mary Jo Foley: Microsoft Aims to Win Student Developers' Hearts with Free Dev Tools. An unblinking eye on Microsoft (web log), zdnet.com, 2008-02-19
Cornelia Koopmans: Microsoft Dream Spark - Free Software Development Tools for Students. LiveSide.net (web log), 2008-02-19
Alfred Thompson: DreamSpark -- Microsoft Gives Software to Students for Academic Use. Computer Science Teacher - Thoughts and Information from Alfred Thompson (web log), msdn.com, 2008-02-20.
Thompson provides a complete summary along with describing its relationship to other initiatives and resources for beginners.
Max Zuckerman: Bill Gates Talks About Free Software, Students, and Technology. Posts (web log), Channel 8, 2008-02-19

As a side matter, it is interesting that Shibboleth and Information Card (they say Cardspace) Identity Providers are supported in the verifying institutions.

[update 2008-02-20: Alfred Thompson's post adds information beyond the basic "hooray" posts and I linked to it. I stick by my recommendation, especially for those who want to learn non-.NET fundamentals.]

DMware: ODMA Futures Roadmap

2008-02-18T22:05:00.001-08:00

Technorati Tags: DMware, ODMA, Document management, middleware

After taking some time to envision what might be possible for an ODMA64 implementation of an Open Document Management API [2], I began to look at further details for a roadmap that could end there. At this point, it looks like there are four overlapping and mutually-supporting stages:

Enhancing ODMA Effectiveness
preserving the sweet spot and improving its support
Preserving the ODMA Core
conserving the historical materials and adding versions that are more appropriate with contemporary tools and libraries
ODMA32 2.5 Integration
surrounding the ODMA32 Connection Manager with higher-level integration layers that happen to permit confirmation of new connection manager releases as well as experimental use of integration points that anticipate ODMA 3.0 simplifications
ODMA64 3.0 Interoperability
breaking with the past to achieve simplified interoperability on 64-bit systems with exclusive reliance on Unicode

Although a few items are slated to commence in March 2008, most of the activity is without fixed calendar commitments and with no detailed deliverables. Here's the current level of thinking.

1. Enhancing ODMA Effectiveness

There are four critical features that have ODMA be effective in bridging desktop software products to document-management systems:

Simple Integration in Desktop-Software User Interfaces
With the cooperation of desktop programs, the ODMA-integrated document management systems present their dialogs as dialog boxes of the desktop programs. This provides direct connection into the flow of information-worker activities. User attention is not taken away from the desktop software. The additional dialogs that accompany creation and use of managed documents are provided automatically. There is nothing for the user to forget to do. This is what users see and what makes ODMA so appealing for using already-available productivity software with managed documents.
Easy Deployment to the Desktop
The ODMA Connection Manager, the enabling middleware, is deployed to office desktop systems along with the ODMA-compliant document-management system client software. ODMA-aware productivity software automatically discover the presence of the Connection Manager; the Connection Manager completes the connection to the document-management system that is to be used in the application setting. The configuration of desktop and document-management connections is managed by ODMA-specific settings in the Windows registry. No software modification is involved.
Trouble-Shooting Support
ODMA 2.0 Connection Manager software provides logging functions that are usable in confirming and trouble-shooting connections between desktop software and document-management integrations. There are also utilities that can be operated to verify correct operation and to experiment with configuration changes.
Low Barrier to Entry for Document-Management Vendors
This characteristic of the ODMA deployment and integration model is appealing for new, small, and specialized document-management vendors. Instead of making difficult per-desktop software (version) custom integrations, the supplier of an ODMA-compliant DMS can be assured that most ODMA-aware desktop programs will "just work" with that DMS.

There is a similar advantage for specialized ODMA-aware desktop software, although ODMA integration is more difficult and takes more preparation when designing desktop software. It is not necessary to build-in any knowledge of individual ODMA-compliant document-management systems, however.

All future development is focused entirely on preserving and enhancing these four qualities. In many cases, it is a matter of providing improved guidance and documentation. Tutorials, supplemental utilities, reference implementations, improved samples, and implementation kits will also be provided over time.

2. Preserving the ODMA Core

The ODMA SDK packages and other materials have become stale on the ODMA Interoperability Exchange site. As part of a general site repaving operation, the core materials of ODMA 1.0, ODMA 1.5, and ODMA 2.0 will be re-organized and preserved on the site and on an anniversary CD-ROM. The CD-ROM will be preserved at AIIM International as well, so that the historical materials are not lost.

Along with preservation of the historical materials, the ODMA32 Core will also be supplemented [3]:

Existing materials will be identified better in the files themselves, and also repackaged for more-convenient separate use. These will also be available on the ODMA Interoperability Exchange site.
The identified and repackaged materials will be made available as downloads of the ActiveODMA SourceForge project. The source code and documentation will be placed on the ActiveODMA subversion source-control system as a foundation for further work.
Some of the developer materials, such as header files, will be supplemented with versions that are more appropriate for development of production ODMA-based programs. These changes will be made in parallel with other activities where the changes can be tested in actual usage. This work has already begun on behalf of ODMJNI 1.0 and it will parallel other ODMA32 2.5 developments.
Existing samples circulated as part of the ODMA Core will be replaced by ones that correct known defects and provide more effective demonstration of the use of ODMA features.
At some point, there may be a maintenance release of an upgraded ODMA32 2.0 Connection Manager. This connection manager will contain some minor corrections and be thoroughly confirmed via automated tests produced before any Connection Manager changes are considered. ODMA-aware applications and ODMA-compliant DMS integrations should see no difference whatsoever. Some areas of ODMA effectiveness will be strengthened.

ODMA Core preservation activities will be initiated in March 2008, following the Public Beta release (0.60) of ODMJNI 1.0. The activity will be carried out in parallel with further ODMJNI releases as well as other ODMA32 2.5 integration activities.

3. ODMA32 2.5 Integration

ODMA32 2.5 is the achievement of supplemental integration layers surrounding ODMA32 2.0. The idea is to enhance the opportunities for ODMA integration with provisions of this kind:

Native COM-based Libraries
There's development of Win32 Native Unicode-based libraries that deliver COM binary interfaces instead of the ODMA API. The use of ODMA32 and the C Language ODMA API is completely hidden. This allows creation of ODMA-aware applications using object-oriented technology.
Managed Code Libraries
The Native COM-based libraries are used to build Java and .NET libraries that can be used to make Java-based and .NET-based ODMA-aware desktop software. If there is any remaining point, an ActiveX library could also be produced.
Native DMS-integration Fixtures
Components needed to build ODMA-compliant DMS integrations are developed as integration kits and also as a basis for test fixtures that allow end-to-end exercise of the ODMA Connection Manager from the Native COM-based Libraries (and above) through to test DMS-integration fixtures below. This creates the possibility of automated regression testing of the ODMA32 Connection Manager. With that possibility, maintenance upgrades to the ODMA 2.0 Connection Manager can be considered.
Managed-Code DMS Integration Interop
Although more difficult, experimental interop layers between Native DMS-integration layers and managed-code implementations (Java or .NET) of ODMA-compliant DMS integrations can be attempted.
ODMA32 Exploration for ODMA 3.0 Integration
As preparation for ODMA64, the native DMS-integration and Interop fixtures might be arranged to allow preparation of DMS integrations that only use features that will be supported under ODMA 3.0 for ODMA64. This will simplify development of ODMA-aware ODMA32 DMS integrations, especially when the Document Management System is already designed around Unicode.

ODMA32 2.5 developments are already underway. Beyond the delivery of the ODJNI 1.0 Public Beta release in March 2008, there is no fixed schedule. The further development and release of ODMA32 2.5 components will be leisurely, paced by ODMA Core continuation work.

The ODMA32 2.5 supplements, along with maintenance updates to the ODMA Core and the ODMA Connection Manager, should continue to be available and usable so long as the Win32 platform remains in use on x86 architectures. No further development is expected for ODMA32 beyond 2.5.

4. ODMA64 3.0 Interoperability

Initial thoughts toward a potential ODMA64 appeared in November 2007 [2]. I am now thinking that creation of ODMA64 for 64-bit platforms will mark the introduction of ODMA 3.0 [4]. ODMA 3.0 will only be supported on 64-bit processors. It will not be backward compatible with the ODMA 2.0, although there will be useful overlap with ODMA 2.5 provisions.

Development of ODMA64 is not expected before 2010. Whenever development does occur, there are three key provisions:

The ODMA Effectiveness qualities are completely preserved (section 1).
Anticipation of ODMA64 will be worked out via ODMA32 2.5 supplements that provide proof-of-concept and also a way to stage ODMA32-based software for migration to ODMA64 by abandoning those features that will not survive into ODMA64.
The ODMA64 Connection managers and supplemental software should be portable to non-Win64 64-bit platforms. There will need to be GUI dependency coordination between ODMA-aware desktop software and ODMA-compliant DMS integrations, but this will be done in a way that does not involve ODMA64 Connection Managers.

I am still bemused that, 10 years after the AIIM ODMA Consortium shipped the last version of the ODMA Core components, based on the ODMA 2.0 specification, I am still supporting this long niche software and its integration model [1]. Not only that, I am now proposing further development in this interesting little middleware laboratory.

[1] Dennis E. Hamilton: ODMA: The Little Middleware that Could. Orcmid's Lair (web log), 2005-12-20.: This article celebrates the 10th anniversary of the ODMA 1.0 specification and availability of the initial middleware for 16-bit Windows and Win32. The AIIM ODMA Coalition ceased formal activity following the completion of the ODMA 2.0 specification and the 1998 availability of the final ODMA Core, still for 16-bit Windows and Win32.
[2] Dennis E. Hamilton: DMware: How About That ODMA64? Professor von Clueless in the Blunder Dome (web log), 2007-11-10. Updated 2008-02-15.: Initial thoughts on taking ODMA into the 64-bit world. I have updated this note to extend the timeline into 2010, allowing more room for polishing the ODMA32 Core and building up the ODMA32 2.5 supplemental integration layers first.
[3] Dennis E. Hamilton: ODMA32 Core. ODMdev Development Note d071004 0.01, AIIM ODMA Interoperability Exchange, February 15, 2008.: This Development Note will be used to track polishing of ODMA32 Core up through any ODMA 2.5 maintenance release of the ODMA32 Connection Manager, followed by any migration to an ODMA64 3.0 core. ODMA 3.0 will not be backward compatible to ODMA 2.0 (or 2.5), using the change of platform to switch over to exclusive use of Unicode and also drop out unsuccessful ODMA 2.0 features.
[4] Dennis E. Hamilton: ODMA64 3.0 Interoperability. ODMdev Development Note d071002 0.02, AIIM ODMA Interoperability Exchange, February 15, 2008.