Category Archives: Research

Epson PhotoPC and EXIF JFIF dates

What you will need:

Once again I find myself searching the internet to solve a particular problem that I seem to be the only one trying to solve, so I will share the problem here, and my solution, in hopes that it may help someone else.

In 1996, I purchased an Epson PhotoPC digital camera. At that time, no one had ever heard of a “digital camera.” I had to call it a “computer camera” when explaining what it was to folks who asked.

Between 1996 and 2000 I used this camera and took thousands of photos as I travelled for work, sharing them on my sites DisneyFans.com (theme parks) and AtTheFaire.com (renaissance festivals). Today, several camera models later, I have well over 100,000 photos online.

In the early days of digital photography, there was not yet a standard to how cameras were going to embed information about the photo inside the JPEG file — such as the date and time the image was captured. With no displays on the early cameras, the only way camera time was set was through a serial cable hooked up to a PC or Mac. In the case of the Epson PhotoPC, it used some form of JFIF header information for the date/time. If you try to view a picture taken by the EpconPC in modern graphics software such as GraphicConverter or iPhoto, it will not show any date information. Apparently, there just is no EXIF data in the file.

I have found that the excellent EXIFRenamer utility for the Mac can parse the JFIF information, allowing files that have it to be successfully renamed and even have the date created filestamp updated.

BUT, I found out too late that early graphics programs would destroy this information when an image was rotated. Modern utilities know about all this stuff and will preserve EXIF data, but back then, it wasn’t well supported.

So now I find myself trying to finally sort thousands of my old personal photos and get everything all updated. A huge portion of my old photos no longer contain JFIF date information (due to being rotated) and none contain EXIF. They might as well be scans from paper pictures!

I began importing my entire picture collection in to iPhoto and noticed immediately the pictures showed no date information — just a date the file was “last modified.” This caused a ton of pics to appear in year 2001 for me. I then proceeded to spend all night trying to manually adjust dates using the iPhoto feature that will do this (a very powerful option, indeed!).

The end result was hours wasted trying to figure out if “0403_004.jpg” was taken in 1997, 1998 or 1999 with some success, and many obvious failures. It was a painstaking process of visual clues — did I have eyeglasses on? Full beard or clean shaven? Which apartment was I living in at the time? After finally getting about 95% “confirmed” (and finding out I have several galleries on my photo sites labeled by the wrong year), I decided there had to be a better way before I tackled my next batch of photos.

So here is my down and dirty solution to this challenge, using iPhoto, GraphicConverter and EXIFRenamer.

  1. Some photos DO contain JFIF date code information, which neither iPhoto or GC can read, but EXIFrenamer can. So, I begin by using a feature of the renamer to rename all images to a temporary name (prepending an “_” to each file) and updating the creation date of the file. Set up a simple custom rename style of %P%o%C%S%F (it really only needs %P for prefix and %O for complete filename, but I use this for other things too):Be sure to set the Processed Files tab to “Change File Creation Date to EXIF Date”:
  2. Rename using EXIFRenamer and this setting. After the files dates have been updated, I then process them through renamer again to get rid of the “_”, setting them back to normal. (I couldn’t find a way to process them without renaming.)
  3. Above, “%1U%F” uses a substring then the file type extension, and below you set Substring #1 to “mid (pos)” with “1” for par1 – this starts from the first character, so it skips the “_” character.
  4. I then use a feature of GraphicConverter to adjust the modification date of files to the creation date.
  5. Lastly, I import things into iPhoto, which will then honor the modification date (it does not seem to care about creation date).
    From here, I can then go back to using iPhoto to adjust the images that are showing up with undefined dates (1969, for some reason, or times in 2001 which must have been the last time I copied all these files over or something).

It’s a huge amount of work, still.

What I Need

I am still looking for a tool that will recursively add EXIF information to JPGS that do not contain EXIF, by using JFIF header information (if available). There is bound to be something out there that does this. Right now, I’m doing the above steps, then using iPhoto to manually set datecode on “missing” photos, which means only they now have EXIF and this process will need to be repeated for the other files if they are ever touched in a way that loses their modification date, or sorted in another tool that relies on EXIF.

Anyone?

Some Screen Shots:

Original image using old file system date:

New image after process (ignore the long filename):

Note that the date is wrong and bogus (1970) but it has at least been updated. For photos that still have correct JFIF date information in them, it will be as accurate as the camera’s timestamp was.

EXIFRenamer settings – two things must be set to do the rename and adjust the file creation date:

EXIFRenamer settings – two more things must be set to create a pattern to rename, but ignore the leading “_” character we just added to all the files:

GraphicConverter – open the folder containing all the images you wish to process, and Apple-A to select ALL, then do a right-click “DATE->Set Modification Date to Creation Date”. (iPhoto uses the modification date, so this extra step is necessary.)

FireWire to SATA adapters for the Mac

My Situation:

I currently have 14 hard drives hooked up to my iMac. I am using three DAT Optic qBox-f enclosures that hold four IDE hard drives, each loaded with 500GB Seagates. These enclosures support FireWire 400 and 800. I hook them to my iMac (which only has FW400) and then daisy chain them through the FW800 ports on the back of the enclosures.

And the end of the chain is two bay enclosure that supports FW400 and USB. That gives me 14 active 500GB drives online for video storage. (I also have a few other 250s and 200s I swap in and out from a single case, when I need to.)

The Problem:

I’m out of disk space, and the newer 1TB-size drives do not seem to be available in IDE. The world is moving towards SATA drives. SATA has been used internally in Macs for years now, as well.

I am also out of desk space, so adding more of these qBox-f enclosures and more 500GB IDE drives isn’t really what I want to do.

The Goal:

I now want to migrate my entire system from 500GB IDE drives to 1TB+ SATA drives. If I wanted to keep my existing enclosures, DAT Optic makes an adapter that will plug on to the back of each SATA drive and give it IDE hookups. At $16/each, though, I’d rather just find new SATA enclosures and sell what I have and start fresh.

The Roadblock:

While two bay FireWire enclosures could be found for $60, and quad bay enclosures for under $200, once you step up to SATA, for some reason FireWire support is massively expensive. I’ve seen quad bay FW/SATA enclosure for well over $400, which is out of the question on my budget.

USB to SATA enclosures are abundant, and can be found for similar prices as IDE models, but USB is more CPU intensive and slower than FireWire. When dealing with multi-track audio (in GarageBand) or video (in Final Cut), I already run into situations where I get the dreaded “too many tracks to realtime preview” errors. So, even if USB only adds 5% more CPU time, and is only 1% slower, that would still impact my workflow.

The Plan:

I plan to find a way to use the FireWire ports of my iMac to hook up enclosures that contain SATA drives.

This is a log of my progress so far.

My Progress So Far:

One of the advantages SATA has is that they defined the port connectors and spacing. This allows for drive enclosures where you can slide a SATA drive in without having to deal with cables. I discovered some trayless enclosures that let you pop open the front and just slide drives in, as if they were cartridges in an old Atari 2600.

I did a search on Amazon and found the v7age220-EF 2-bay enclosure made by iStarUSA. It could hold two SATA drives, and hook up via FW800 or USB. It was also trayless, meaning you could open up the two doors on the front and just exchange drives, without using any tools.

The problem was the price – $179. Sure, it had more indicator lights, and supported some RAID stuff (without having to use a disk utility to reprogram the case, like some of my others), but that was over $100 for what I’ve found similar IDE enclosures.

The company selling the drive through Amazon was GoHardDrive.com. They had it directly from their site at $149 with free shipping, so I went ahead and placed an order. Unfortunately, after a week, I checked and noticed it still hadn’t shipped, and the product page now said they were out of stock. I contacted GoHardDrive and they apologized, and cancelled my order.

Instead, I purchased the USB version of this case – the v7age220-SAU. Instead of FW/USB (like the -EF model), the -SAU model supported USB and eSATA. And, it was only $85!

So, at least according to iStarUSA, having FW adds about $80 to the cost of an enclosure. I’m not sure I buy that, since even CompUSA (high price retail store) sold single-drive FW/USB enclosures for less than $60 (for the entire enclosure and power supply).

I received the v7age220-SAU and it worked great. It was quiet, nicely built (except for the front doors which seem pretty flimsy) and fast. But, I still wanted a way to use FireWire.

FireWire to SATA – The Quest Begins:

While I was able to find tons of USB-to-SATA adapters dirt cheap, at first I couldn’t find anything that did FireWire-to-SATA.

Eventually, I discovered two converter boxes sold by CoolDrives.com. I have not been able to find these sold anywhere else on the Internet, which is kind of bothersome.

The first is a tiny box, currently sold for just $30, that adapts USB or FireWire to a SATA port:

http://cooldrives.stores.yahoo.net/sata-to-firewire-usb-mini-converter.html

The second model was similar, but had an additional FW pass through port, so you could daisy chain. It sold for $60 (which seems unreasonably high just to add a second FW port):

http://www.cooldrives.com/satofimiad2x.html

The $30 version said it only worked with drives up to 500GB, while the $60 model claimed 1000GB (1TB). I wrote to ask if this was true, and CoolDrives.com said they would have to check. (I haven’t heard back yet.) This means the first model is instantly not usable with the modern size drives (currently shipping 1TB and 1.5TB drives) which seems odd.

But, my idea was, I could get one of these, then plug up a cheaper SATA-only enclosure, like the iStarUSA v7AGE420-ES 4-Bay Trayless eSATA RAID Box External Hard Drive Enclosure for only $165! Now we’re talking! I could actually get a better (?) four drive enclosure that was then use a cheap $30 adapter box. The end result would be FireWire performance from the iMac to cheaper SATA high capacities drives.

But would that work? The answer was NO, and I’ll explain more in a moment…

Also, a co-worker pointed me to this device: FirmTek SeriTek/SpyderHUB

It was designed for this very purpose – to hook a Mac with FireWire up to SATA enclosures. It had two SATA outputs, and accepted both FW and USB as input. But, the adapter alone was $129. It also noted it was not compatible with “Port Multiplier” enclosure. This was a new term to me, and I did some research.

Port Multiplier is the magic that lets one SATA connector then go off to multiple drives. The iStarUSA enclosures are PM devices. In JBOD (just a bunch of disk) mode, each of the drives inside would appear as a separate hard drive, but only if the SATA controller on the computer supported PM.

Long Story Short: (too late!)

It turns out, the $60 box from CoolDrives.com does not support PM devices. Plugging it up to the v7age220 USB/SATA box only shows up the top drive. (I speculate that if I ran these enclosures in a RAID or BIG mode, they would appear as one massive device to the computer, but I have not tried. I do not want this. If one drive fails, I don’t want it taking out my whole array. I’d rather have separate drives I can back up separately.)

Solution:

None yet, but if you can deal with the performance of USB, getting a hub and some of these iStarUSA enclosures may be a decent way to go (but it will cost more than similar IDE enclosures would have).

Future:

I have done extensive benchmarking of drive performance using USB to SATA adapters, the CoolDrives FW/USB adapter, and running straight to the iStarUSA enclosure (all to the same Western Digital WD1001 drive).

I will eventually be posting all the results in a nice table here (or maybe a YouTube video of all this).

If you found this page, and this is interesting to you, please drop me an e-mail. Also, if you have better answers or solutions, please let me know.

Thanks!

References:

iStarUSA – makers of the enclosures. They have yet to return my e-mails requesting information on who sells the FW models.

GoHardDrive.com – discount seller of the iStarUSA enclosures. They have been very responsive to my e-mail requests, but when I ordered, the item stayed listed as “Processing” for a week! When I checked the last day, it suddenly was “Shipped” and tracking showed it was expected that day. They actually shipped it the same day I ordered, but the log did not reflect that.

CoolDrives.com – The only place I’ve found the FW to SATA adapter. I read much bad about them, but my experience was just fine. They shipped immediately and I had no issues.

FirmTek – Makes of a Mac USB/FW to SATA converter. Note they designed this to work specifically with enclosures they sell, and do not promise it will work with any other enclosure. I have no experience with them, but if they’d build in PM support and keep the price similar, I’d buy one today!