While there are lots of consumer-level products to manage digital images, photographs and the like, institutional options, it seems, are not that plentiful. Moreover, those options that do exist tend to cost a pretty penny. I don’t mean Adobe “Album” and Picassa. They’re wonderful products; I use Picassa myself. It’s great for an individual, but not so great for an organization.
[Here's another in my continuing critiques of the scalability and limits of most web-two-dot-oh stuff. It's part of my ongoing litany about nonprofits needing "the basics" – It ain't all about fancy web sites, sophisticated communications systems, and interactive bloggy-wiki-groovy things. The real world needs working infrastructure, including training and training and training, and accounting systems, and communications systems, and shared calendars, and to not keep their mailing lists in a word document, and, and, and… … more than anything, staff that know how to use the stuff. Boring, I know, but true. ]
What got me thinking about this was two-fold: First, I have a crying need to manage thousands upon thousands of images — a collection that dates back some 75 years, and second, someone asked the very question over on the “InteRider” listserv, to wit:
“Does anyone have any recommendations for a digital photo management system for a small NGO? We are looking for something that makes cataloguing, indexing, storage and retrieval of images relatively easy and efficient.”
There was, of course, a flurry of responses — including folks suggesting both Picassa and Flickr — but all suffered, in my opinion, from the same old problem: These damn things are not designed for organizations.
Let’s look at Flickr, for example. I love it. I love it enough to pay for the “Pro” account. I load my pictures up there on a regular basis. Feel free to browse my latest set, entitled “Brussels Noir” – my photo safari through the night streets of Brussels one gloomy February evening or two.
However Flickr — multi-user from the get-go — still suffers the faults of a consumer product. It is a consumer product, so it’s not really a fault. Flickr tools, for example, are designed to help you get items uploaded and into your Flickr site. But, I challenge anyone to point out the tools that let you get photos out of Flickr [aside from the old, right-click-and-save-as]. This is not a product for managing large collections of photographs within an institution. It’s especially not a tool for finding, selecting, editing, manipulating, and generally mucking about with the images. Damn.
Having no where else to turn, I turned to myself. I built a photo management system with SharePoint. I call it a DAM system — DAM is short for Digital Asset Management. It’s fun to talk about in meetings. I get to say things like: “The DAM system works” or “You need to update the DAM subject codes.” Great fun, believe you me. [Somewhere, still inside of me, is that 12-year-old Boy Scout that got such great joy out of singing "Amster, Amster - Dam, Dam, Dam" around the campfire. Strange, I only remember the chorus, not the rest of the words.]
My DAM system is a testament to the flexibility of SharePoint. Start to finish, once I had it spec’ed out, it took only a few days to build — and most of that was in thinking thru the supporting tables and coding systems and then loading up those tables with real data. [Now, in interest of full disclosure, I had thought about the problem for a couple of years, and I had designed a couple of other DAM systems in previous versions of SharePoint, including SharePoint 2001, SharePoint 2003, and WSS 3.0. – but it was still pretty easy.]
The building (and the spec’ing) required some backwards thinking: the ephemeral art of imagining the goal and then working backwards to figure out how you got there. Truthfully, it helps to think backwards. Something I’m good at.
My first foray into programming was on a 200 step programmable HP calculator. It was in high school, amazingly enough. The “calculator” was size of a large microwave, it cost $25,000. You could pick the equivalent today as trade-show swag. It was programmed in “RPL” – short for Reverse Polish Lisp. The emphasis was on the “Reverse” – to program this beast, you had to know the answer and work backwards to the problem. I learned then I was good at assuming a goal and working backwards.
With database design, the same things hold true, despite the intervening years. I just think backwards. Between you and me, with SharePoint, backwards thinking is required. It’s either that, or you end up redoing things a couple of times. In the end, it’s easier to have all the support tables setup before you set up your master list or document/picture library.
Building a DAM System:
First (or is it really last?), I needed a few supporting tables (or as SharePoint calls them, “lists”). My design goals included:
- Keep it simple, yet powerful.
- Where possible, use forced vocabularies (read: taxonomy) to catalog the images based on pick-lists and lookup tables.
- Provide an easy to use, consistent, error-free, and end-user-proof system for cataloging.
- Provide “Google-like” tools for search, ’cause that’s what users will tolerate.
- Provide simple tools to sort, manage, organize, view, retrieve and repurpose images.
Hence, my design called for lots of pre-populated lookups. I’d let the anarchists and other folksonomic folks frolic elsewhere in the design. They get their fields, I get mine. And, since I’m the designer, I’d make mine “required” [evil laugh].
I decided on a total of six supporting tables, one each for subject, program,
location, credit, geography, and usage. I describe them and how they’re used, below:
- Subject (Coding): Multiple choice list of pre-selected (controlled) subject terms. I used a subset of the NPC since we already use this taxonomy within the organization. I built the list with two fields and concatenated those fields to create a single field for the lookup. The first field was a “code” in the form like “A10″ or “E4″ and the second the term itself, like “aquarium.” Using the “code” lets you sort by the code instead of the description, hence logically grouped terms tend to stay in the same area of the lookup table. Less messy and easier to use.
- Program (Coding): This is a completely idiosyncratic taxonomy used to catalog our grants. This is so engrained within our thinking that we see the world through these lenses whether we wish to or not. Again I used two fields and concatenated them into a single lookup much like the subject codes, in form: “Program” — “Program Area.” For example, “Environment—Freshwater” or the like.
- Geographic Focus (Coding): This is a three-tiered hierarchal coding originally based on ISO country codes. The table allows up to three fields that are, again, concatenated into a single lookup field. However, in this case I concatenate in reverse order to make the pick-list easy (e.g., “Ann Arbor – Michigan – United States” instead of “United States – Michigan – Ann Arbor”). I also include a “Special” category, such as “Great Lakes – Special.” This lets us have ad-hoc groupings for regions, special areas, and the like. People tend to think “ad hoc” and this solves that problem.
- Credit (Supporting): This is a list of photographer’s names and other contact information. Since it was “people information” I got smart and just used a standard MOSS “Contacts” list. It had everything I needed and more.
- Location (Supporting): This is a completely idiosyncratic list of hardcopy storage locations. Since we have a lot of original photographs, stored in our archives, it’s a reference field to the actual physical location. If we only have it as a digital asset, the option is DAM (as in Digital Asset Management).
- Usage History (Supporting): This is a list of publications where the photo may have been used. Not strictly germane to the system, it’s a user requirement. We use it to track when and if we used the photo in a printed publication. There is also an entry for “Web Site” and the year. Seemed useful.
Here’s a snapshot of how the various supporting tables look like:
Table 1: DAM Geographic Codes
Table 2: DAM Program Codes
Once I had the coding tables and supporting tables in place, building the DAM system was relatively easy.
And, it was here that I got creative and designed in the folksonomic flexibility that I knew people would ask for. I was pretty sure that I had designed in enough structure — enough forced vocabulary — that the system could survive the ensuing chaos. So I went wild (in a database designer way) and added in free form “Keyword” and “Description” fields. I have a suspicion that the “Keywords” field will prove redundant, given that the search engine indexes all fields, it strikes me that the “Description” field should be more than sufficient. Moreover, given user-fatigue, I imagine that we will quickly stop using all but the required fields, and “Description.” Simple always wins.
Trying to think simple (and backwards), the entire system has only 11 metadata fields, in addition to the system fields. I outline the fields below.
| DAM Custom Fields |
SharePoint Provided Fields |
| Item Title | Thumbnail (automatically generated) |
| Program Code (Lookup/multiple) | Check In Comment |
| Keywords (free text) | Content Type |
| Subject Codes (Lookup/multiple) | Copy Source |
| Geographic Focus (Lookup/multiple) | Created (date) |
| Description (Free text) | Created by (username) |
| Item Location (Lookup) | Date Picture Taken |
| Usage History (Lookup/multiple) | File Size |
| Credit (Lookup) | File Type (extension) |
| Copyright Statement | ID (unique record ID) |
| Modified (date) | |
| Modified by (username) | |
| Picture Height | |
| Picture Width | |
| Picture Size (K) | |
| Type (Icon) | |
| Version (if you run versioning) |
Most, if not all, of the system fields are automatically created and populated by SharePoint — this keeps to the design criteria of “simple” yet powerful.
DAM User Interface:
SharePoint provides a sparse user interface OOTB (out of the box); lots of potential, but not much “substance” on the screen. Without some customization, MOSS pretty much drops you to an almost-blank screen. Users don’t like dropping on a blank page. SharePoint leaves you wondering. Unless you put them on the page, any document libraries, or picture libraries, or other content is kind of “hidden.” I remedy that by creating what I call “a soft landing” — usually something simple like a nice graphic that folks can click. The click takes you into meat of the system.
Tool-wise, MOSS gives you a whole toolbox at your fingertips. You get thumbnails, a “slide sorter” view, a list view, and tools to upload, download, and insert into Outlook, or Word, or open in an editor. One thing you can say about Microsoft products is they integrate well – ah, as long as it’s with other Microsoft products.
Figure 1: A DAM Soft Landing
For this DAM system, I used actual content for the landing page –a picture for the link into the photo repository, and a graphic for the link into the graphic repository. To make it easy to change things in the future, the two “click boxes” on the front page are simple “CEWP’s” (Content Editor Web Parts) that reference to generic graphic images named “PhotoLogo.jpg” and “GraphicLogo.jpg.” The images can be swapped out easily by simply placing replacing the graphic files. The JPG files are stored in an unlisted photo library.
(Oh, I forgot to mention I did a similar system for graphic images – uses the same codes, mostly. The only real change was to include artists in the “Credit” table, and a few different supporting tables. But, this post is long enough without me going into that piece of the pie.) Note the nice navigation on the left side – a list of the supporting tables and codes. These are, of course, security-trimmed — only showing if you have appropriate rights, etc.
Clicking on logo of the “shoes” takes you into the DAM system. [See, ain't that fun.] There are two views predesigned views — one designed as a “list” showing a small thumbnail, and allowing you to filter on the various metadata fields. This is how I expect most people to wander around in the system, filtering down on a particular subject or geographic code; and a “Slide Sorter” view. Here’s a picture of the pictures – two different views:
Figure 2: DAM List View
Figure 3: DAM Slide Sorter View
Now, the tools that SharePoint brings to this, OOTB, are formidable. First, everything is indexed by the MOSS search engine. And, just so you don’t feel left out, this works in WSS too. There is nothing here that can’t be done in the bundled version of Windows SharePoint Services (WSS 3.0).
Moreover, within either of these views, you get access to a range of actions, including upload, download, edit, and send to, as well as the ability to subscribe to the library itself via RSS or via MOSS’s own “Alert Me” system that works via Outlook.
Figure 4: DAM Actions
Finally, the Send To feature bears mentioning. It’s a neat little menu that lets you pop selected pictures into Office apps. I used it to insert the following screen cap, right from the picture library and into this blog post.
The fact that all of this can be done fairly quickly and easily with a product that is bundled for free with Windows Server 2003 is pretty damn neat — and that doesn’t even begin to touch it’s flexible, integration with Word, InfoPath, and the rest of Office. Damn, DAM: damn neat.
Nice! I love software that you can quickly configure into doing really powerful stuff like this.
What about using Adobe Portfolio? A robust, very powerful and well tested DAM? ACDSee – another tool?
Did you try to compare your system with those?
To certain extent I am often wondering what drives people to break through already open door?
Hi Wojtek,
I’m not one that likes to, as you say, “break down open doors.” I did look around before I chose to build my own battering ram. I actually looked at both ACDsee (I’ve used it personally for years), and at Adobe Portfolio. Both are good examples of what I didn’t want.
Here’s why:
ACDSee is a consumer product – a damn fine one — but I needed something for an enterprise environment, one where 30+ people might be contributing, cataloging, or otherwise mucking around with the photographs; and many more searching, viewing, etc, the photos. ACDSee is a stand-alone product; neither right nor scalable for my needs.
Portfolio, on the other hand, would be perfect. Moreover, a single user license is relatively affordable at $199. Unfortunately, it is also one of those options that cost a pretty penny when you try to scale it to an organization. The server license is $6K and that’s without any client licenses. Client licenses are $313 each. At that price, it’s not an option.
Then, finally, I’m a strong believer in (re)using the tools one has on hand. I already have a significant investment in SharePoint – in software and in client licenses, but more importantly, in people skills — given this previous investment, the overall ease of development, and amazing flexibility of SharePoint, in the end, it was an easy decision.
In other words, I wasn’t breaking down an open door; I was just walking through the open door that was right in front of me.
P.S. your photos are gorgeous.
Gavin,
Is it possible for someone to contact you about your DAM system regarding purchase?
Gavin,
Were you able to set up security levels? For instance, people could view but not download without permission?
Hi Aileen:
The answer is: yes.
’cause this is all in SharePoint, the security is controlled by SharePoint. YOu can set up all sorts of levels of security, including read-only.
However, of note, since this is a database of pictures, and people can therefore VIEW the pictures in a browser, it’s almost impossible to stop them from using the “right-click” to save a picture to a file…. but Sharepoint has a very flexible security system.
Hi Gavin,
I own a small 5 person home building company here in Houston Texas. At age 31, i have pushed my company to be technology focused and driven where efficiencies apply. We have just implemented Share Point 07 and need a way to manage all of our photos within. Like you, we are EXTREMELY cost concious as we are in our third year of business and trying to grow. Is there a way to download your awesome tool for us to integrate to our Share Point 07 implementation? I would really appreciate it, if there is a cost, please let me know.
I see you have Texas roots…
Lee Z
Hi Lee,
I’m more than happy to share.
Here’s the scoop: I can export a copy, stripped of any photographic content. It can then be imported into an existing MOSS installation using the STSADM tool. You’d need to enter your own categories and codes and the like. I’ll provide a brief set of instructions.
Now all I have to do is figure out how to post it somewhere for download. I’ll get back to you, or contact you directly, when I figure it out.
as to the roots, they’re deep:
My father was a Texan
My grandfather was a Texan
My great grandfather was a Texan and a Texas Ranger — captured Bell Star, so they say.
Gavin-
Let me know if you need a place to post this. I’m sure we could put it up on one of UW’s servers, and this is exactly what I’ve been looking for. In fact, colleagues and I were just lamenting the need for this again yesterday, which led me to find this thread.
Hi folks,
I think I’ve figured out how to post this … but I ran into a snafu.
I designed this originally in MOSS 2007 — I took it and stripped out all of my data, leaving empty tables for you cataloging pleasure.
However, just to double check the portability of the exported template, I tried to import it into a clean, standard WSS installation.
It kind of broke, something to do with the search functions, I believe. Consequently, I’m going to create a stand-alone WSS version and then upload them both. This will take a little time. I should have it in a week or so. It will be a MOSS or WSS template (.stp) file.
Gavin
Nice job on your tool. One thing for people to consider is the cost of their own time. I’m often surprised out how often I hear someone say, “I couldn’t find something affordable so I built it myself”. Your time is valuable and you need to factor that in to the cost equation! So, unless you are in the business of selling DAM tools, anytime you spend building a tool is time that you are not spending growing your business.
If you own or work for a small/medium size business, your time is probably your most important asset. It is not uncommon for a small business owner to put a value of $300/hr – $500/hr on their own time. And, even when you use a cool tool like Sharepoint, this stuff will take time to put in place. Beyond the initial programming, you need to factor in time for maintaining servers, enhancing functionality, upgrading system software, etc.
But, if you can’t find a tool that has the exact functionality you need, it may be necessary to build your own.
Aquent (who I work for) is now offering a tool that I think meets many of the requirements you discussed. The tool is called MajorTom http://www.majortom.net and it was designed to be an affordable digital asset management application that can scale to the needs of global orgainzations.
Gavin,
Did you ever set your DAM up for download anywhere? I would love to give it a try as well.
By the way, “Chris” over at Aquent (posted above), this is not a place to sell your crappy MajorTom DAM. Any company that places security high on its priority list wouldn’t use your hosted solution. This blog is about a Sharepoint DAM integration!
Eric
Hi Gavin,
Nice article, and I like your Brussels photos, it is such a terrific city.
I was thinking of using SharePoint in the same way for our org’s DAM needs :-)
One feature I’m struggling with is how to keep a couple of different resolutions of an image grouped together. Have you dealt with that all in your experience?
Other than that, a feature we may include is to put some workflow on top so that certain people in the org can be notified when new assets of a certain type get uploaded.
Regarding your re-disribution effort with WSS, the searching will probably be heavily crippled. Good luck.
Aaron,
I solved the resolution issue by not addressing it.
Seriously, instead of trying to work out a way to store multiple copies of the same pictures, in different resolutions, we adopted a policy of storing only the highest resolution. Then, using the tools built into both MOSSS and Office 2007, we’ll “downgrade” a photo as needed.
From within MOSS/WSS, on the “Actions” toolbar, you get a choice of “sending” pictures to either open applications and/or Office applications. Within that dialog, you can specify the “original” size, or lots of variations, from 1024×768 on down to 160×160 pixels.
Gavin
We use Erez (http://www.yawah.com/products/erez.html) at work.
It can scale ridicilous large pictures, bundle downloads of multiple pictures and zoom extraordinary large pictures without the viewer knowing it.
You can see it in action on the Erez homepage or my companies use here: http://www.rmig.com/sw769.asp?lsearchdataid=33089
Gavin,
I’ve been looking for some inspiration in making some SharePoint media gallery / multi-user blog/photo site, and your post is the closest thing I’ve found for storing/viewing large amounts of pictures through SharePoint.
I noticed you mentioned exporting the .STP file. Have you been able to share out that file? I would definitely jump at a chance to download and install it for personal use.
Let me know if you can send/post, would be awesome!
Thanks!
Oops, after wanting to see what other cool stuff you had up here, found your link for the .STP on the main page.
Thanks, will give it a try!
[...] The same is true for Picasa. (See my post about my Digital Asset Management (DAM) system, here and [...]
I would try VeriPic. http://www.veripic.com
They have been in use with police departments for quite some time due to the high volume and security needs. They are now in use (as far as I know) in some companies and hospitals, along with military installations. I hope that helps! – Drew
Hi Gavin,
i am the guy from Texas that you responded to last year about your DAM solution. You were going to upload it to an FTP for me, but i think got too busy. Would you still be able to do so, we are in bad need of this solution and would actually like to implement it in the next day or two if you can upload it for us? I would be GREATLY in debt to you and thank you so much in advance.
Lee
713-443-1134
leezieben@zkhomes.com
Lee,
Check the post entitled: “Wham, BAM, DAM” at “http://www.digitaldiner.org/2007/08/22/wham-bam-dam/
I posted a link that doesn’t expire. Still works, still up there, all yours.
What about “MediaRich for Sharepoint”. The only truly integrated DAM system I could find “out of the box”. We blew through $50k in development/consulting fees and had nothing useable for Digital Asset Managment workflows inside of Sharepoint 2007. Then we tried (they have a sandbox) and downloaded (easy to install into Sharepoint 2007) Equilibrium’s MediaRich for Sharepoint. Loved it! 23k later, we have something that reads in just about all media and services 200 people for self service integrated DAM in Sharepoint. If I had known about this sooner, I would have saved alot of time and money! It adds advanced metadata search, video ingestion, video info views, all office files are thumbnailed, has great exporting and batch processing of media (very cool). They even fixed the uploader to handle huge uploads from any machine (we have a bunch of macs in the marketing department). Everyone is raving…I love it when something ads so-much functionality to a Microsoft platform that’s very hard to develop on and makes the IT department look like we are miracle workers without the work!
You should highly consider SmugMug from a photo sharing perspective. Has all the things you are not finding in Flickr, plus the ability to pull out the exact image you uploaded (bit perfect) and to do some fairly powerful metadata management.
Plus it’s hosted and has redundant data centers, negating the need for any local “CPE” hardware.
Hi, just stumbled across your article whilst looking into exacrtly the same thing for a team sharepoint site i’m working on. Unfortunately lots of your pics don’t display – any chance you could replace them back again :)
Inspiring stuff anyway !!!
Hi Matthew,
Thanks for letting me know about the pic problems. The fine folks at the Gilbert Center were working on blog … looks like we’ve got it sorted now. (The pictures should be there now.)
Regards
Thank you so much. This just saved about 4 weeks of time for me. You are a scholar and a Texan.
I am working with a church here in friendswood, Texas. They have a huge number of photographs that they wish to have an online catalog of going back several years. Based on your Blog I am assuming that an SQL server and a Sharepoint Server would more then be sufficent. My last database (I am an infrastructure engineer) was Dbase and clipper. Are there any suggestions for a start point to bui;ld a system such as you describe herein.