While you may order that inflatable, remote-controlled zeppelin online, the acknowledgement nevertheless comes via email, as does the receipt, and the shipping updates.

Email is the truck that moves freight – light and heavy – on the information-super-goat-trail. Plain, simple, elegant, boring, your-grandma-has-an-AOL-address-type email remains the venerable heavy lifter of the online world.

Strangely, it has also become the de facto identity management tool. It is universally used to authenticate just who we are, on everything from my bank to the myriad of social and anti-social real-time networking sites. When we forget just who we are, it’s the delivery method of choice to jog the memory or to trigger a reset — ironically, given how totally insecure it really is, likened to a postcard.]

But, the core problem with email is not security. The real problem with email is it’s really stupid. It’s dumb as a bucket of overripe bananas. I mean it. It’s really god-awful stupid. It can’t help it. It was designed that way.

When push comes to pull, with email, you really don’t get much, and that illustrates its frailty and its amazing functionality.

With email, you see, all you really get is an “envelope” (consisting of minor variations on “From,” “To” a “Subject” along with tiny little bits of routing data that nobody pays attention to) and a giant blob of undifferentiated stuff called a “message body.” What’s in that message body is anything, unstructured, and undifferentiated – a blob.

What’s in there could be secret silly croonings to your one-true love, it could be the confirmation of your getaway flight to a land without extradition, or it could be my secret recipe for the world’s best gazpacho (and hence your necessary and immediate flight from justice).

With email, the medium hides the message. (If I keep this up, I’m likely to be haunted by McLuhan.)

Since its launch in 1971, we have improved it. We’ve tweaked it and twiddled it. We’ve made it better, making it easy, for example, to stuff it with bits of binary. We’ve said goodbye and good riddance to Uuencode and its ilk. Now digital civilians needn’t know a MIME type from a mime troupe. We’ve prettied it up, too — love it or hate it — with HTML, providing that ever-so-useful ability to deliver ugly fonts, in all the sizes, shapes, and colours your little heart could desire, rendering it pretty much unreadable

[For the record: I think I sent my first in the fall of 1979, using a service called EIES. I've got a copy of it around here someplace. It was a message to the fellow at the desk next to me, suggesting we get lunch at the Burrito King (tacos al carbon); truly important stuff!]

And, so, email moves the world, moving commerce and confirmations, in the wink of an eye. We’ve filled the tubes with everything from solicitations for various dysfunctional systems (whether erectile or congressional), to orders for green tea, multiple drafts, rewrites and painful iterations of your latest annual report, and those important PDF copies of your sinister plans for global domination through puppy adoption. They’re all sent on their respective ways via email.

Email is the go-to tool for everything from “donuts in the kitchen,” to presidential elections. But, inside, it’s still one dumb blob.

Blob, meet the software equivalent of Steve McQueen: Email2DB– one magnificent tool.

Steve McQueen (saving diners) in the Blob!

Email2DB has become a necessary cog in my machinery. It lets me take those dumb blobs and ferret out the necessary bits and pieces of the message, shaping them, cleaning them up, adding value in terms of structure, and then, gently slipping that data into a giant database.

In the best of all possible worlds, I wouldn’t have to do this. In that world, I wouldn’t be tasked with figuring out what to do with thousands email messages sent willy-nilly to just about any email address, person, or inanimate object you might care to imagine. But I do.

I am charged with capturing and organizing thousands of inquiries — inquiries that arrive in every way imaginable, some via a web form, others via email, and still others via such unspeakable things as (shudder) fax. I even think a few get slipped under the door at night by pixies. They’re all important, and regardless of origination, I want them all to end up in the same place — a database. Despite their disparate origins, I want them all channeled into the waiting, eager programmatic minds for review. I am all about the smooth flow of information. My motto: Never, ever, type it twice.

Email2DB keeps me true to my motto. In a nutshell, Email2DB “deconstructs” the email. It breaks it into its constituent parts, slicing and dicing the blob, parsing not only the header, but the contents, and gently slipping those deconstructed pieces into the database of your choice – in my case, the same-same database used to capture the content entered via a fancy online web form. Derrida would be proud.

Well known parts of the email —like the TO, FROM, SUBJECT, DATE — as well as some of the arcane bits and pieces of the underlying protocol (Originating IP, MessageID, ReplyTo) are a breeze to deconstruct.

They’re “pre-programmed” into the software, and with a single mouse click, you can pull those wee bits apart and slide them into a database. It talks to all-comers: Access, SQL Server, Oracle, MySQL, Access, ODBC, yada yada yada. It’ll even write it out as a CSV if you’re living in 1996.

Once you’ve extracted the pieces you can use them for nefarious purposes: perhaps to construct a new message, sending back, for example, custom acknowledgments, or forwarding on reformatted confirmations, or simply adding them into a database for further processing. Email2DB takes email and turns the contents into fields and records in your favorite database.

The Taming of the Blob -- Smart Parsing for Dumb Blobs

Fancy stuff is easy. Within an hour I was rolling my own routines to parse more bits and pieces from the message, isolating the “First name” and “Last name” from the so-called “Friendly Name” portion of the “From” field. “Don’t stop there,” I said to myself.

So, I tackled the blob itself. With a little head-scratching, a smattering of OOP concepts under my belt (and a passing familiarity with VB), I was able to deconstruct bits of the body of the message itself, scanning through the text for familiar references that might match my mighty database elements.

With only a little fancy footwork, I was even able to detach any attachments, saving them with a unique “key” to a SharePoint document library, along with a PDF copy of the original message (also tagged with the same unique key).

The beastie will read and process messages from POP3, IMAP, and Exchange servers. It will also read and process messages directly from Outlook folders, including Exchange “Public Folders.” It has a fairly full-featured scripting language, and variables, once created are reusable.

While this is all well and good for me and mine, the beauty of this product is its universal application. There’s not a week goes by that someone on doesn’t ask me for the easy way to get information from a web site to a database. While there are a myriad of ways — some are easy and some are not. None are as easy as email.

Moreover, if it’s email generated by a web form, you control the structure. If the structure is predictable, Email2DB can easily grab that email, work with your structure, find the right bits and pieces, deconstruct them into the raw data you need, and then, easily slip that deconstructed data into an eagerly awaiting database. All is right with the world.

The requirements are minimal. The Email2DB software costs $300, $500 or $1,000, depending on features. I went with the $500 copy, as I needed the scripting engine and attachment processing. You need an email account (any will do, including those reached via SSL). Finally, I run it on a virtualized XP machine, rigged to autostart, autologin, and autorun, should anything interrupt its dedicated rounds.

Overall, the customization took about three days. I created scripts for:

• pattern matching to extract first and last names
• file-renaming to save copies of the original message and attachments to a SharePoint library
• Unique (per message) tags so that saved items could be retrieved as a group
• URL constructions so the database could include links to the original message and attachments
• Other custom flags for the source, date and time received, and type of inquiry

Whether you’re looking to manage the email meteor shower, stave off an invasion of unstoppable email blobs, or just want to turn a few dumb ones into smartly structured data, Email2DB can do it. It’s not often you can find software that will not only stop an alien invasion, but will also send you an acknowledgement when it’s done. Steve McQueen not included.

I’m back, and now I’ve knocked “sharing the DAM thing” off my “to-do” list. Read on.

Gavin’s DAM System

Finally (finally!), my brain kicked into gear. I figured out an easy way to share the DAM thing. Moreover, I re-did the whole thing in WSS just to make sure it was totally portable and didn’t have any residual MOSS attributes, etc.

I knew it should be easy. I think I was just over thinking it. Finally, one day, I, not so gently, slapped myself in the head and said (to myself): “why not just put it up on one of those free file sharing/storage systems that have sprouted up all over the place in the last few years. Might just be a good chance to take a look at how they work anyway.” A couple of aspirin later (for the headache after the slapping) I did it.

Here’s how: I closed my eyes and picked one. Once I picked one, I then puttered around with it trying to figure out how to actually share something. Then I trashed it, and tried another. Trashed that one, and tried yet another, and then another, and … finally I found one that would do what I wanted. It’s a nice service and it seems to work. It’s called File123. It seems workable.

The downside — it has a time limit. Four weeks. We’ll work around that simple enough. As long as people want it, I’ll keep resetting the date and/or posting a new copy.

So, (tad da!) you can find the completed DAM WSS template here:

http://www.file123.com/Transfer.aspx?guid=33a3935c-a4e4-4df0-975b-f066fcfe34e7

(Link updated 10/29/2007 – the new link will expire on 11/26/2007)

(12/4/2007)

I got tired of updating that link, so here’s one that won’t expire, up on Microsoft’s new (beta) Skydrive. Click here.

As I said, that URL will work for four weeks (all these systems impose some sort of time limit – I guess so you don’t use them as some sort continual, free, FTP hosting service just like I am doing). In four weeks I’ll re-share the file and reset the date.

Now: Some information:

First, I re-wrote the whole thing for WSS. My original (and working) design is in MOSS. But, I discovered that while something designed for WSS works just fine in MOSS, the reverse is not necessarily true. So, I just redid it in WSS.

Second, I completed the “Graphics” section. The photograph section was fully formed, but the Graphics section had been left wanting. I also put in some sample graphics and pictures. Of course, feel free to delete.

Third, I’ve saved it as a WSS template, instead of a “backup.” This, I believe, makes it much more portable and flexible. Everything is still there (I chose the option of saving data with the template), but templates are designed as “re-usable,” so there should be no issue with broken links or the like. I also eliminated the “search” portions. Instead, it should fall right into whatever search is appropriate for your particular installation.

Fourth, here are some basic instructions

Step 1: Add the template to your MOSS/WSS installation:

• Choose: Site Actions
• From the Galleries list, choose: Site Templates
• From the List menu, choose Upload and then upload the DAM WSS STP file into your site gallery.

Step 2: Choose to create a new site. On the site information page, choose the template under the CUSTOM tab in the Template Selection named “DAM WSS Version.”

Finally — and I say this seriously — download and have fun. However, Caveat Emptor and all that jazz. I offer no guarantees. I distribute this free of all responsibility and liability, use and/or abuse at your own risk. It should give you no problems, but if it does, well… I’ve left town, yeah, and am now living on the island of Fernando Poo, or maybe the long lost Isles of Langerhans… Yeah, that’s it.

The trouble was even though it had a setting for “current user” buried deep in some of the “filter” options; it returned the information in the form “Domain\LOGON_USER” or the common name (Firstname Lastname). I never could figure out how to easily extract just the portions I needed and to use that portion in any of the off-the-shelf web parts or data views. As I said, perhaps it was just ignorance.

Instead, I ended up going the long way round and built out some special code to make a connectable web part that could be used to filter by username. It was a lot of work. And, of course, it broke in the transition from SPS2003 to MOSS 2007. Connectable web parts are great — but this was just too much work.

Consequently, in this version, I vowed to do something simpler. I knew it could be done; it was just a question of beating my head against it until I either:

A) beat myself senseless and forgot what I was doing, or

B) found the keys to the kingdom.

I suppose there was always option “C) Hire a consultant” but my experience with consultants, with a few notable exceptions, has not always been that satisfactory. Consultants seem to like to borrow your watch to tell you what time it is. The good ones, of course, then keep the watch.

The goal: simple, a stand-alone, self-filtering web part that displays data based upon the current user. If I did it right, the code should be reusable, or at least easy to duplicate.

I’m happy to report that I succeeded. Moreover, once I figured out where to do it, it was easy — and it opened up a whole passel of possibilities. (And I vowed to write it down so I wouldn’t forget it. I’m keeping my promise.)

About DVWPs:

Before I talk about “the how,” let me digress a minute to talk about “data views” or “data view web parts.” They’re the other half to this puzzle, and they’re real special, magical even. They are one of the more powerful SharePoint tools. Affectionately called DVWP’s, these little beasts do magic. And they’re right cute too. Like love, and like all magic, it’s kind of hard to explain unless you see it; even then, you’re not too sure how it works.

Please note: To play with DVWPs you need SharePoint Designer (MOSS 2007) or Visual Studio. The cost of admission is worth it. I use SharePoint Designer, but beware it is buggy as hell and seems to crash all the time. You get used to it, and learn not to do some things in some sequences. Such is life or Microsoft or both.

DVWP’s display data from databases such as SQL Server (or any ODBC-compliant data source) on a SharePoint page. In a nutshell, DVWPs give you a table of data, like a data grid, or a form. It’s that easy, by the way, to set up a web-front end to a database, for both viewing and editing.

If you’re looking to build some sort of dashboard, aggregating existing data, and displaying it dynamically on a web page, well, here is one easy way to do it.

Once set up, you can display data; you can shape it, mold it, and filter it. With the flick of a mouse, you can put it inside of handy dandy expanding and contracting headers, or sort it, or change the format or the display, based upon any of the values in the data itself. You can hide things or expose things, or change their color and format, based upon the values in the table. It’s just damn neat — and easy.

Here’s a simple sample (below), a DVWP showing “open” helpdesk support tickets dynamically pulled from our helpdesk system called TrackIT. Total Time to create: 2 minutes. I’ve redacted names to avoid tormenting the paranoid.

I warn you now, that while creating and tweaking DVWPs is easy as pie, the tough part is figuring out where to stop. All of a sudden, all that data that’s locked up in LOB databases can be easily displayed on a web page — without code, without anything other than (some) knowledge of the data structure and a connection to the SQL server.

As I mentioned, you need SharePoint Designer. From there, it’s easy. Just a few menu clicks, and some mousing around. I’m not going to cover them here, but, trust me, they’re phenomenally easy. Next stop, filtering. Buckle your seatbelts. The management is not responsible for headaches, eye strain, or slaps to the side of your head.

Before we go any further — and before you tell me that there are already ways to filter in MOSS — let me explain a bit more. You’re correct: built in to MOSS is a parameter called [Current User]. Unfortunately, for my purposes, it doesn’t work. It returns the current user in the form “First name Last name” (e.g., John Smith). We don’t use that. (D’oh) Where it’s appropriate, we tag data using the so-called USERNAME, in the form “JSMITH” (first initial concatenated with last name). It’s ubiquitous to all our databases. I know, because I made it so. Standards! I’m just a jeepster for standards. [You have to remember T-Rex for that to make any sense at all, not that it makes any sense even if you know the reference.]

So, I needed to work some magic — pretty simple magic if you have access to any sort of programming language — but I wanted to do this without opening the hood. How it’s done turns out to be simple, it’s done with XSLT, and I show you the code, and tell you where to put it. [I don't mean that the way it sounds.]

Personalizing a DVWP:

I’m going to use a real-life example here. I’m going to walk you thru creating a web part that displays data from our Grants Management system — specifically information about employee charitable giving, and the organization’s matching contribution. Our goal is a simple, stand-alone, portable web part that would display data filtered by the current user, in this case, me. What we want is a list of charitable gifts, sorted and grouped by year, showing the amount, the match, and the name of the organization.

What we’re not going to cover how to set up DVWPs or connections to a database. That’s pretty self explanitory.

The data is in the database, all we have to do it get it out, format it on the screen, and filter it by the Windows Server variable “LOGON_USER.” In the end, it should look like this: Again, I’ve redacted so you don’t see my gifts to folks like the Rosicrucians, the L5 Society, and the Center for Epistemological Inconsistencies and Eccentricities (I’m a charter member!).  Obviously, I’m an equal opportunity push-over.

Pretty simple really. You might note, I got a little fancy. I use some simple DVWP tricks put up a little red “new” flag if the gift date is less than a month old.

Here is how it’s done:

• Step 1:     Set up a connection to the database using the DVWP wizard.
• Step 2:    Select the table or view you want, and select the appropriate fields.
• Step 3:    Insert the data into an appropriate web part zone, and move the items around, until you get them the way you want them.
• Step 4:    Set up the groupings and headers and all that jazz.
• Step 4:    Set up the filter and sit back and admire your work.

By the way, while you’re working with a DVWP, I recommend you adjust a couple of settings right away. To get to the settings (and where you work the magic) click the little “menu” arrow thingy on the right side of the DVWP. Here’s a before and after shot, showing the task pane.

I set the various defaults as follows:

1. Click the box labled “Show with sample data” — this will populate the fields with sample data. This is important, especially if you use a filter that has no current results.
2. Set the the Data View Preview to hide all filters. Otherwise, if you filter out all the results, you have no current results.
3. The “Sort and Group” option lets you pick sort fields, and select to group on a certain field of set of fields. This is also where you set the expanding/contracting header groupings.
4. Notice the menu item labeled “Parameters.” This is where we set up the dynamic variable that pulls in the LOGON_USER.
5. Finally, we’re going to tweak the first parameter, the one labled “Filter.” This is where the magic happens.

Now we’re ready for the filter.

First we need to set up a “parameter” — click the menu item on the “Common Tasks” menu (above) labeled “Parameters.” You should see the following window:

We’re going to set up a parameter called “LogonUser” with the following attributes:

• Name:     LogonUser
• Parameter Source:    Server Variable
• Server Variable name:    Logon_User

When you’re done, it should look like the screen above. What this gets us is a dynamic parameter we can use elsewhere in the DVWP. It contains the full Windows Logon_User string, in the form “Domain\Logon_User.”

Next we’re going to turn it into something we can use by stripping off the domain and the slash (\) and converting it all to upper case. We do this right in the filter statement.

As shown above, from the “Common Data View Tasks” list, click the “Filter” menu option. It’s the first one on the list. You should see the “Filter Criteria” dialog box. It’s probably blank, and looks much like this:

Note the little check box at the bottom of the window. This is where we add the filter magic. Check the box, and then click the button to edit the XSLT filtering. Ignore the birght yellow warning that says: ”Using XSLT Filtering can reduce performance.” This is just a typical “If-you-do-this-your-naughty-bits-might-just-fall-off” warning. I just ignored it. Nothing fell off yet.

Finally we get to the filter – shown below in all its glory! Yep, that’s it. That whole line of gobbledygook. I’ll explain what it does, working from the inside out. (And quit your squinting – I put the final formula below so you can just copy and paste it into your DVWP.)

Constructing the Filter (from the inside out):

In the examples below, I show variables and formula in blue, and the current value of that statement in reddish brown. We start first in the center of the gobbledygook, with the $LogonUser parameter.

$LogonUser = Domain\JSmith

In the center of the expression is the parameter we created, called LogonUser, only now it is prefaced by a “$,” indicating that it is a parameter. This evaluates as the string “Domain\JSmith”

(Substring-After(“Domain\JSmith”,”\”) = JSmith

Surrounding the LogonUser parameter is a function to extract a substring from the parameter, using the “\” as the starting place in the string. The function extracts all the text following the “\”. We now have a parameter with just the name, e.g., JSmith.

Translate(JSmith’),’abcdefghijklmnopqrstuvwxyz’,'ABCDEFGHIJKLMNOPQRSTUVWXYZ’) = JSMITH

Now that we have the logon_user name handy, we have to convert it to upper case. XSLT is really clumsy here. Instead of a simple “UCase” statement or some such, we have to use a “Translate” function to convert every occurrence of a lower case letter to an upper case letter. Luckily I only need to do this once, and now I’ll just cut and paste it.

@Staff_ID = JSMITH

Finally, we can use an “Equals” operator to compare the variable @Staff_ID (pulled from the data source, to the LogonUser parameter we’ve constructed (or deconstructed, as the case may be).

The full statement is shown below: (note I’ve broken the line into two for display purposes). The brackets and everything else is important.

[@Staff_ID = Translate(Substring-After($LogonUser,'\'), 'abcdefghijklmnopqrstuvwxyz','ABCDEFGHIJKLMNOPQRSTUVWXYZ')]

That’s it. If you run Gifts, for example, and use the same format for STAFFID, you should be able to take this stuff verbatim and drop it into any part you set up.

Since that foray into Betatown, MS released a new WHS version, the so-called “RC1″ or “release candidate 1″ edition — just a little bit closer to an actual commercial release. Just between you and me, I don’t pretend to follow Microsoft’s various mystical and mysterious machinations as it (slowly) walks a product to market. RC1, or Beta 3, or CTP, they’re all beta’s to me. It means you can’t buy it, yet. It also means install at your own risk. A beta, by any other name, is still likely to break your heart, eat your hard drive, and maybe shred your collection of precious Godzilla DVD rips.

While I am on the subject of both the mystical and Godzilla — has anybody noticed that Microsoft has a severe case of marketing schizophrenia? At times, it’s almost comical.

Why, on one hand, with WHS, are they introducing network file shares for the home market, while simultaneously encouraging migration away from shares (to things like SharePoint) in all other markets? Why didn’t they just bundle SharePoint Services (WSS) with WHS? They could have called it a “break-through in web-based home file storage” and garnered lots of hype. It makes a whole lot more sense than their decision to introduce a line of furniture in the form of a table-sized PC. Gack! That thing only serves to remind me of wasted hours with the lovely Ms. Pac Man.

Mystical musing aside, I was nevertheless eager to once again take another walk on the seedy side of Betatown, so I installed RC1. (Everything was running just fine, so it seems a perfect time to screw things up.)

The upgrade itself was uneventful, smooth even; smooth enough to make me nervous. So far: “Everything is beautiful and nothing hurt.” This also seemed an opportune time to take a look at the “add-ins” feature of WHS.

What are they? Well, they’re little server based processes — that do things. The one I specifically wanted to see was something called PhotoSync. It supposedly automates uploads to Flickr from a designated WHS folder or folders. I thought that sounded like a neat idea. I’ve been having lots of fun lately with Flickr. (Shameless self-promotion alert: I’m especially fond of this one I call “Looking for the heart of Saturday night,” with apologies to Tom Waits.)

The concept behind PhotoSync is simple: You put photographs in a designated WHS folder and it will automagically post them to your Flickr account. I tried it. It worked. It was nice.

The down side: it didn’t do tagging, at least not yet. But it will automatically create new photo sets. That’s a nice feature. Moreover, it will automatically title the photographs based upon the name of the file. All in all, a nice piece of work, that.

On the side, what’s also nice is the “add an add-in” process itself. (Try saying that fast.) It’s brain-dead simple. You need only copy the appropriate installation files to the appropriate folder on WHS. Moreover — in a true break from tradition — the appropriate folder was even logically named (\Software\Add-ins).

When an add-In is copied to the right folder, it shows up on the WHS management console. You click the button to install, or, if it’s already installed, you click the button to un-install it. Simple.

Unfortunately, there appear to be relatively few add-ins available — maybe four or five max. Nevertheless, they are all “community developed” and freebies, which is pretty cool. You can find them at a place called We Got Served. I note that there is a handy µTorrent client, and some sort of media extender that will stream media (music, DVDs, videos, photos) called WebGuide.

Meanwhile, while I was doing all the requisite staring into space that goes hand in hand with any software installation, I got to pondering… a dangerous thing, that. When I ponder, I get ideas — wild ideas, fanciful ideas — sometimes they lead me on a journey of discovery, other times I just end up breaking things.

What I got to pondering about was SharePoint and Windows Home Server. I got to thinking about the fact that WHS was just a stripped-down version Server 2003, the OS required for Window SharePoint Services (the free version). To me, it seemed such a shame, such a waste of resources — to have a whole server doing not much but humming to its self. What a waste of processing power, and resources, and potential.

Now, clearly I could install Seti@home, feel good about my carbon footprint or some such, and simultaneously help search for intelligent life in Washington, but I had a better idea. [Besides, I think that's fruitless; alien life, sure, absolutely, but Washington, not likely at least for another 18 months; highly doubtful even then.]

Instead, I pondered up this idea: Why not build a Frankenstein —and… I could even name the server “Abby Normal” — I could transplant a WSS brain into WHS body! Clearly I have watched too much Mystery Science Theater 3000. Just as clearly, I’m entertained by simple things.

Anyway, my logic went like this: if it’s Server 2003, why can’t it run SharePoint too? Besides, it’s free. I could even use it when I wanted to do some SharePoint development at home and, well, who knows, it might be handy. Besides, what have I got to lose except all my backups?

The fact is, I could imagine a nice WSS/WHS Frankenstein, err … Fronkensteen, as being very handy for your typical nuclear, post-nuclear, and not-so-nuclear family.

In my mind — strange as it may be — the typical features of a small “team” site might be quite applicable to both the home market and the small NGO market — shared calendar, “cork” board, shared file storage, etc. Having a quick and dirty WSS Intranet happily humming on WHS might just be the ticket.

It also fit in to my plans to write up a couple of SharePoint “use case” studies I was thinking of calling “Gavin’s Five-Minute Guide to SharePoint Intranets.” (Coming soon to a theater near you.)

“What a grand idea,” I said to myself. So, I set out to see if it could be done.

Give my creation life!

The short story is: yes, it can be done. In fact, it was easy as pie. First you need a brain…

Seriously, there was only one small SNAFU, and that was easily solved with some quick tweaks to IIS (Internet Information Server) — changing a few ports around so that all the various web sites would work. Below, I lead you through how to do it and how to avoid the SNAFU.

Here are the simplified steps:

1. Login to WHS as administrator and ignore the big splash screen warning that your naughty bits are going to fall off if you mess about with stuff. Relax, really. Trust me <evil grin>.
   
2. Grab your mouse and prepare to mess about. Close the warning window — out of sight, out of mind — and open IIS Manager.
   
3. In IIS Manager, locate the “Default Web Site” in the Web Server’s list, and change the port from 80 to 81. Leave the SLL port (443) the same. If you want to know why, I explain what this does at the end of this posting.
   
4. Open “My Computer” and create a folder on the root of the [D:] drive and name it WSSIndex. We’ll use it later to hold the WSS index files and avoid filling up the limited space on [C:]. (WHS sets up a rather small [C:] partition, so I figured not to crowd it.)
   
5. Download WSS if you haven’t got it already, it’s here. Then double-click the install file (it’s called “SharePoint.exe”)
   
6. Accept the EULA, ignoring the fact that you’ve probably just offered up your firstborn to Satan.
   
7. At the dialog window where you choose an installation type, choose the option for “Advanced.”
   
8. On the next dialog window, click on the “Data Location” tab, and switch where the beast stores its index files. Change it to the folder you created on the [D:] drive, e.g., D:\WSSIndex.
   

   
   

9. Go back to the “Server Type” tab, select the option for “Stand-Alone” and press the “Install Now” button.
   
10. Make a cup of tea and pet the dog for a while as it installs. Consider making a political donation or moving (back) to Canada, or both, or maybe Scotland.
    

    
    

11. When Windows is done whirring and installing, click the “Close” button, making sure the checkbox for “Run the Configuration Wizard” is checked. This will start the WSS configuration wizard.
    
12. Click “Next” to tell the wizard to keep on keeping wiz’ing, and accept the silly warning that a few services will be reset. It will run through all sorts of tasks — ten in all, so…
    
13. Make another cup of tea. Check your Gmail to see if anybody has written you lately. Sigh, sadly, at all the spam, close Gmail and go back to work.
    
14. If all goes well, about this time, Windows SharePoint Services should now be installed, and you should see something like this:
    

    
    

15. Click “Finish” … and if all went according to plan, you should be dropped at the default WSS web page. It should look sort of like the picture on the left (below). If you substitute “HTTPS” for “HTTP” you should see the default WHS site, as shown on the right (below).
    

You’re done, by the way. You’ve now got a SharePoint site to play with. Both the WSS and WHS web sites should be available at URLs shown above, from anywhere on your home network (substitute your server name, of course).

WSS itself is administered through the Control Panel, under Administrative Tools, and click the item labeled “SharePoint 3.0 Central Administration.” The first thing to do is add authorized users to the WSS site — if you’ve set up WHS with local users, you can use the same login credentials (user accounts) on WSS. You add them right under site actions, site settings, “People and Groups.” Easy as pie.

Have fun!

About Port 80 and Port 81:

I mentioned above that I’d talk briefly about why we moved one web server from port 80 to port 81. Port 80, as you no doubt know, is the default web server port. What we did is free-up port 80 for WSS. It simplifies the installation process, as the WSS wizard is going to use Port 80 no matter what. But, if we leave the WHS in place, on Port 80, the WSS installation wizard ends up shutting down the server on both port 80 and port 443 (SSL/HTTPS). By changing the default server to port 81 before hand, we avoid that problem, and the server on port 443 is left untouched.

It’s easy to do.

1. Open IIS Manager and locate the Default Web Site.
   
2. Right click and choose Properties. You should see a window like this:
   

   
   

3. Change the TCP port from 80 to 81 and click OK.
   
4. Right-click on the default web server and choose stop. Wait a few seconds and then right-click again, and choose start. That usually gets everything humming nicely again.
   

This way, it keeps everything neat – otherwise, the installation just stops the WHS instance and leaves it stopped. Sure, there are other approaches, including host headers and adding additional IP addresses to the network interface. But this is the simplest and has no impact that I can see.

Quite frankly, I don’t think you need the WHS web server on port 80. All that web site does is redirect you to the SSL port (443). It’s simpler and easier to just use HTTPS for your WHS server and leave WSS as is, on port 80. So far, it’s worked just fine for me.

[Here's another in my continuing critiques of the scalability and limits of most web-two-dot-oh stuff. It's part of my ongoing litany about nonprofits needing "the basics" – It ain't all about fancy web sites, sophisticated communications systems, and interactive bloggy-wiki-groovy things. The real world needs working infrastructure, including training and training and training, and accounting systems, and communications systems, and shared calendars, and to not keep their mailing lists in a word document, and, and, and… … more than anything, staff that know how to use the stuff. Boring, I know, but true. ]

What got me thinking about this was two-fold: First, I have a crying need to manage thousands upon thousands of images — a collection that dates back some 75 years, and second, someone asked the very question over on the “InteRider” listserv, to wit:

“Does anyone have any recommendations for a digital photo management system for a small NGO? We are looking for something that makes cataloguing, indexing, storage and retrieval of images relatively easy and efficient.”

There was, of course, a flurry of responses — including folks suggesting both Picassa and Flickr — but all suffered, in my opinion, from the same old problem: These damn things are not designed for organizations.

Let’s look at Flickr, for example. I love it. I love it enough to pay for the “Pro” account. I load my pictures up there on a regular basis. Feel free to browse my latest set, entitled “Brussels Noir” – my photo safari through the night streets of Brussels one gloomy February evening or two.

However Flickr — multi-user from the get-go — still suffers the faults of a consumer product. It is a consumer product, so it’s not really a fault. Flickr tools, for example, are designed to help you get items uploaded and into your Flickr site. But, I challenge anyone to point out the tools that let you get photos out of Flickr [aside from the old, right-click-and-save-as]. This is not a product for managing large collections of photographs within an institution. It’s especially not a tool for finding, selecting, editing, manipulating, and generally mucking about with the images. Damn.

Having no where else to turn, I turned to myself. I built a photo management system with SharePoint. I call it a DAM system — DAM is short for Digital Asset Management. It’s fun to talk about in meetings. I get to say things like: “The DAM system works” or “You need to update the DAM subject codes.” Great fun, believe you me. [Somewhere, still inside of me, is that 12-year-old Boy Scout that got such great joy out of singing "Amster, Amster - Dam, Dam, Dam" around the campfire. Strange, I only remember the chorus, not the rest of the words.]

My DAM system is a testament to the flexibility of SharePoint. Start to finish, once I had it spec’ed out, it took only a few days to build — and most of that was in thinking thru the supporting tables and coding systems and then loading up those tables with real data. [Now, in interest of full disclosure, I had thought about the problem for a couple of years, and I had designed a couple of other DAM systems in previous versions of SharePoint, including SharePoint 2001, SharePoint 2003, and WSS 3.0. – but it was still pretty easy.]

The building (and the spec’ing) required some backwards thinking: the ephemeral art of imagining the goal and then working backwards to figure out how you got there. Truthfully, it helps to think backwards. Something I’m good at.

My first foray into programming was on a 200 step programmable HP calculator. It was in high school, amazingly enough. The “calculator” was size of a large microwave, it cost $25,000. You could pick the equivalent today as trade-show swag. It was programmed in “RPL” – short for Reverse Polish Lisp. The emphasis was on the “Reverse” – to program this beast, you had to know the answer and work backwards to the problem. I learned then I was good at assuming a goal and working backwards.

With database design, the same things hold true, despite the intervening years. I just think backwards. Between you and me, with SharePoint, backwards thinking is required. It’s either that, or you end up redoing things a couple of times. In the end, it’s easier to have all the support tables setup before you set up your master list or document/picture library.

Building a DAM System:

First (or is it really last?), I needed a few supporting tables (or as SharePoint calls them, “lists”). My design goals included:

1. Keep it simple, yet powerful.
2. Where possible, use forced vocabularies (read: taxonomy) to catalog the images based on pick-lists and lookup tables.
3. Provide an easy to use, consistent, error-free, and end-user-proof system for cataloging.
4. Provide “Google-like” tools for search, ’cause that’s what users will tolerate.
5. Provide simple tools to sort, manage, organize, view, retrieve and repurpose images.

Hence, my design called for lots of pre-populated lookups. I’d let the anarchists and other folksonomic folks frolic elsewhere in the design. They get their fields, I get mine. And, since I’m the designer, I’d make mine “required” [evil laugh].

I decided on a total of six supporting tables, one each for subject, program,
location, credit, geography, and usage. I describe them and how they’re used, below:

• Subject (Coding): Multiple choice list of pre-selected (controlled) subject terms. I used a subset of the NPC since we already use this taxonomy within the organization. I built the list with two fields and concatenated those fields to create a single field for the lookup. The first field was a “code” in the form like “A10″ or “E4″ and the second the term itself, like “aquarium.” Using the “code” lets you sort by the code instead of the description, hence logically grouped terms tend to stay in the same area of the lookup table. Less messy and easier to use.
• Program (Coding): This is a completely idiosyncratic taxonomy used to catalog our grants. This is so engrained within our thinking that we see the world through these lenses whether we wish to or not. Again I used two fields and concatenated them into a single lookup much like the subject codes, in form: “Program” — “Program Area.” For example, “Environment—Freshwater” or the like.
• Geographic Focus (Coding): This is a three-tiered hierarchal coding originally based on ISO country codes. The table allows up to three fields that are, again, concatenated into a single lookup field. However, in this case I concatenate in reverse order to make the pick-list easy (e.g., “Ann Arbor – Michigan – United States” instead of “United States – Michigan – Ann Arbor”). I also include a “Special” category, such as “Great Lakes – Special.” This lets us have ad-hoc groupings for regions, special areas, and the like. People tend to think “ad hoc” and this solves that problem.
• Credit (Supporting): This is a list of photographer’s names and other contact information. Since it was “people information” I got smart and just used a standard MOSS “Contacts” list. It had everything I needed and more.
• Location (Supporting): This is a completely idiosyncratic list of hardcopy storage locations. Since we have a lot of original photographs, stored in our archives, it’s a reference field to the actual physical location. If we only have it as a digital asset, the option is DAM (as in Digital Asset Management).
• Usage History (Supporting): This is a list of publications where the photo may have been used. Not strictly germane to the system, it’s a user requirement. We use it to track when and if we used the photo in a printed publication. There is also an entry for “Web Site” and the year. Seemed useful.

Here’s a snapshot of how the various supporting tables look like:

Table 1: DAM Geographic Codes

Table 2: DAM Program Codes

Once I had the coding tables and supporting tables in place, building the DAM system was relatively easy.

And, it was here that I got creative and designed in the folksonomic flexibility that I knew people would ask for. I was pretty sure that I had designed in enough structure — enough forced vocabulary — that the system could survive the ensuing chaos. So I went wild (in a database designer way) and added in free form “Keyword” and “Description” fields. I have a suspicion that the “Keywords” field will prove redundant, given that the search engine indexes all fields, it strikes me that the “Description” field should be more than sufficient. Moreover, given user-fatigue, I imagine that we will quickly stop using all but the required fields, and “Description.” Simple always wins.

Trying to think simple (and backwards), the entire system has only 11 metadata fields, in addition to the system fields. I outline the fields below.

Most, if not all, of the system fields are automatically created and populated by SharePoint — this keeps to the design criteria of “simple” yet powerful.

DAM User Interface:

SharePoint provides a sparse user interface OOTB (out of the box); lots of potential, but not much “substance” on the screen. Without some customization, MOSS pretty much drops you to an almost-blank screen. Users don’t like dropping on a blank page. SharePoint leaves you wondering. Unless you put them on the page, any document libraries, or picture libraries, or other content is kind of “hidden.” I remedy that by creating what I call “a soft landing” — usually something simple like a nice graphic that folks can click. The click takes you into meat of the system.

Tool-wise, MOSS gives you a whole toolbox at your fingertips. You get thumbnails, a “slide sorter” view, a list view, and tools to upload, download, and insert into Outlook, or Word, or open in an editor. One thing you can say about Microsoft products is they integrate well – ah, as long as it’s with other Microsoft products.

Figure 1: A DAM Soft Landing

For this DAM system, I used actual content for the landing page –a picture for the link into the photo repository, and a graphic for the link into the graphic repository. To make it easy to change things in the future, the two “click boxes” on the front page are simple “CEWP’s” (Content Editor Web Parts) that reference to generic graphic images named “PhotoLogo.jpg” and “GraphicLogo.jpg.” The images can be swapped out easily by simply placing replacing the graphic files. The JPG files are stored in an unlisted photo library.

(Oh, I forgot to mention I did a similar system for graphic images – uses the same codes, mostly. The only real change was to include artists in the “Credit” table, and a few different supporting tables. But, this post is long enough without me going into that piece of the pie.) Note the nice navigation on the left side – a list of the supporting tables and codes. These are, of course, security-trimmed — only showing if you have appropriate rights, etc.

Clicking on logo of the “shoes” takes you into the DAM system. [See, ain't that fun.] There are two views predesigned views — one designed as a “list” showing a small thumbnail, and allowing you to filter on the various metadata fields. This is how I expect most people to wander around in the system, filtering down on a particular subject or geographic code; and a “Slide Sorter” view. Here’s a picture of the pictures – two different views:

Figure 2: DAM List View

Figure 3: DAM Slide Sorter View

Now, the tools that SharePoint brings to this, OOTB, are formidable. First, everything is indexed by the MOSS search engine. And, just so you don’t feel left out, this works in WSS too. There is nothing here that can’t be done in the bundled version of Windows SharePoint Services (WSS 3.0).

Moreover, within either of these views, you get access to a range of actions, including upload, download, edit, and send to, as well as the ability to subscribe to the library itself via RSS or via MOSS’s own “Alert Me” system that works via Outlook.

Figure 4: DAM Actions

Finally, the Send To feature bears mentioning. It’s a neat little menu that lets you pop selected pictures into Office apps. I used it to insert the following screen cap, right from the picture library and into this blog post.

The fact that all of this can be done fairly quickly and easily with a product that is bundled for free with Windows Server 2003 is pretty damn neat — and that doesn’t even begin to touch it’s flexible, integration with Word, InfoPath, and the rest of Office. Damn, DAM: damn neat.