| |||||||||||||
unfortunately this misdesign is commonunfortunately this misdesign is commonPosted Jun 17, 2010 4:31 UTC (Thu) by JoeBuck (subscriber, #2330)Parent article: A quick grumpy review of Shotwell There seem to be any number of photo applications that want to suck your photos into some internal database somewhere, often by copying all of the images, which could eat up several gigabytes of disk without warning and commits you to using only that app. I don't know why people think that this is a good idea.
(Log in to post comments)
unfortunately this misdesign is common Posted Jun 17, 2010 5:28 UTC (Thu) by eru (subscriber, #2753) [Link] I agree, but what are the alternatives if you want the tool to provide more organization than just whatever is in one directory? The tool might keep a database with metadata and references to the original pictures (by file name), but the links get broken if the pictures are moved (eg the user adds more disks and decides some photo directories need to relocate).But whatever the solution, it should be straightforward and as much self-documenting as possible, since we are storing data for years or decades, and migrations to different hardware and software are a certainty. I'm currenly not using a photo management software but my plain directories full of pictures I might want to find later are getting larger, so I have been thinking about what would be a good organizer. I thought of Shotwell, but our Grumpy Editor just lessened my enthusiasm for it.
Very true but it begs the question of generic picture management policy. Posted Jun 17, 2010 5:39 UTC (Thu) by NicDumZ (guest, #65935) [Link]
> The tool might keep a database with metadata and references to the original pictures (by file name), but the links get broken if the pictures are moved (eg the user adds more disks and decides some photo directories need to relocate).
Very good point. Maybe one could have a hash-based and/or distance-based wizard to rebuild a broken database when the tool launches and detects missing files and cant trivially find the pictures matching their name?
Is this the time when someones steps forward and says "we need a standard for managing media collections", and asks all existing tools to support such standard? It seems that some common base is the only way to have different tools collaborating correctly with each others...
Very true but it begs the question of generic picture management policy. Posted Jun 17, 2010 6:10 UTC (Thu) by doogie (subscriber, #2445) [Link]
git-based, and git notes?
Very true but it begs the question of generic picture management policy. Posted Jun 17, 2010 13:45 UTC (Thu) by dgm (subscriber, #49227) [Link]
Just my thoughts. Git is supposedly a dumb content tracker, so it looks like it's just asking for it.
And btw, I didn't knew of git-notes, thanks for the pointer!
Very true but it begs the question of generic picture management policy. Posted Jun 17, 2010 17:42 UTC (Thu) by dlang (✭ supporter ✭, #313) [Link]
to make this work well in git you would need to make a diff function that would handle the EXIF metadata appropriately (so that if you change a EXIF tag git doesn't think the entire file is different)
this isn't a huge undertaking, but I'm not aware that anyone has done it yet.
Even after you do this you may have grief with large repositories.
Very true but it begs the question of generic picture management policy. Posted Jun 20, 2010 20:09 UTC (Sun) by nix (subscriber, #2304) [Link]
For images? Unless you can convince everyone to rescale their photos to small sizes (good luck with that), it's not going to work. git isn't good at giant binary lump tracking.
(e.g. I've been taking photos for less than three months in a serious way and I already have >10Gb of them. git repos that size are hard indeed to manage.)
Very true but it begs the question of generic picture management policy. Posted Jun 21, 2010 1:07 UTC (Mon) by dlang (✭ supporter ✭, #313) [Link]
this partly depends on if the changes to the files are in the images or if it's mostly changing metadata.
if it's a lot of editing (and saving the intermediate images) git is definitely bad for the job.
if it's mostly metadata changes and tagging, then git is merely poor (with a fairly clear roadmap of how to get better)
Very true but it begs the question of generic picture management policy. Posted Jun 21, 2010 21:33 UTC (Mon) by nix (subscriber, #2304) [Link]
Unfortunately, even if there are very few changes to the images, git does badly with giant binary blobs. It's true that does *worse* if the images are edited heavily, but, really, for things like this you don't want *any* extra copies of the images lying around, and git's going to give you one every fifty-odd commits, as long as there's a modification in there somewhere.
Very true but it begs the question of generic picture management policy. Posted Jun 17, 2010 7:16 UTC (Thu) by ekj (guest, #1524) [Link]
That is, infact, PRECISELY what kphotoalbum does.
It stores meta-information about the files in a xml-file, including the sha1sum of the files. This means that if you move, reogranize or rename your files, the meta-information will still apply.
The meta-information -also- sticks if you edit the file, because it has filename AND sha1sum. But if you do both, you lose the meta-information, unless you run "update sha1sums" (a maintenance-tool available in the tool-menu)
So "edit + update sums + move" is fine. But "edit + move" results in lost metainformation. Reasonable, in my opinion, because if the FILE changed AND the filename changed, how is any tool supposed to know it's "the same" file?
Kphotoalbum also does NOT import your pictures, nor ever change them in any way (though it supports kipi-plugins, and some of those do modify files)
Very true but it begs the question of generic picture management policy. Posted Jun 17, 2010 9:43 UTC (Thu) by eru (subscriber, #2753) [Link] Thanks for bringing it up. I'll have look at kphotoalbum, it sounds like the kind of tool I am looking for.
Very true but it begs the question of generic picture management policy. Posted Jun 17, 2010 13:02 UTC (Thu) by dmag (subscriber, #17775) [Link]
+1
I've been using kphotoalbum for years. The interface could be more intuitive, but it's OK once you get used to it. It's got hierarchical tags, add/remove multiple tags, etc. It will bring your recently-used tags to the top (but there is still a way to use a tag w/o bringing it to the top). It's got multiple categories of tags (people, places, keywords). It automatically finds any new photos you throw in the photo directory. (Let's face it, all your photos should be in one directory. 1TB drives are like $60 now.)
But my biggest wish would be for a really good web app, so my family could view and help tag the photos.
Very true but it begs the question of generic picture management policy. Posted Jun 17, 2010 12:43 UTC (Thu) by jackb (subscriber, #41909) [Link]
Sounds like yet another problem that a filesystem which supported attaching arbitrary metadata to files accessible via standard file/directory semantics could have solved.
Very true but it begs the question of generic picture management policy. Posted Jun 17, 2010 16:47 UTC (Thu) by martinfick (subscriber, #4455) [Link]
Until you move your files to another filesystem! What you are proposing simply glosses over the problem by calling the internal DB the local metadata enhanced filesystem.
Very true but it begs the question of generic picture management policy. Posted Jun 17, 2010 18:29 UTC (Thu) by jackb (subscriber, #41909) [Link]
I didn't propose it - Hans Reiser did.
Had reiser4 fs been completed as initially intended the required functionality would have been added to the VFS layer and more than likely ported to other file systems by now.
Then the "how to we store and access metadata" problem would be solved permanently and universally. (at least on Linux)
Very true but it begs the question of generic picture management policy. Posted Jun 17, 2010 18:55 UTC (Thu) by tzafrir (subscriber, #11501) [Link]
Until you need to back it up. Or copy it to Windows.
Very true but it begs the question of generic picture management policy. Posted Jun 20, 2010 20:28 UTC (Sun) by nix (subscriber, #2304) [Link]
Or access it over the network ($HOME on NFS -> you can hardly use it at all).
Very true but it begs the question of generic picture management policy. Posted Jun 21, 2010 11:59 UTC (Mon) by jond (subscriber, #37669) [Link]
indeed, as soon as I burn my photos to backup DVDs, bye-bye metadata.
Very true but it begs the question of generic picture management policy. Posted Jun 28, 2010 20:53 UTC (Mon) by bjartur (subscriber, #67801) [Link] It's a simple tradeoff:
Very true but it begs the question of generic picture management policy. Posted Jun 29, 2010 11:22 UTC (Tue) by paulj (subscriber, #341) [Link]
Why not develop a file format that can handle multiple streams internally? Then you get the best of all worlds:
- works with tools which think files are just files
There are a number of such formats already, what we're lacking is a format that is optimised for multiple-streams for data storage.
Very true but it begs the question of generic picture management policy. Posted Jun 29, 2010 16:41 UTC (Tue) by nix (subscriber, #2304) [Link]
Wouldn't we call these multiple-stream files 'directories'?
Very true but it begs the question of generic picture management policy. Posted Jun 29, 2010 17:45 UTC (Tue) by paulj (subscriber, #341) [Link]
Yep, but that's fragile in the face of applications that are not aware that the dirs+files are arranged in certain way - leading to the problems the parent poster perceives which caused them to argue for xattrs (which are much, much more fragile in the same sense - and hence much worse).
Very true but it begs the question of generic picture management policy. Posted Jun 29, 2010 20:03 UTC (Tue) by nix (subscriber, #2304) [Link]
FUSE (automounted by a library) plus something like tarballs (only not actually .tar, I know .tar is vile, maybe .gz.cpio, i.e. gzipped first then cpioed?) might do the trick.
Very true but it begs the question of generic picture management policy. Posted Jun 29, 2010 18:15 UTC (Tue) by dlang (✭ supporter ✭, #313) [Link]
you mean like EXIF tags inside a image file?
Very true but it begs the question of generic picture management policy. Posted Jun 30, 2010 6:59 UTC (Wed) by paulj (subscriber, #341) [Link]
Yes. Is EXIF limited in some way though, that makes it less than useful for image apps to put all their metadata in there?
Very true but it begs the question of generic picture management policy. Posted Jun 30, 2010 7:19 UTC (Wed) by dlang (✭ supporter ✭, #313) [Link]
probably the fact that it's fairly expensive to search metadata that's part of many separate files compared to having the data condensed into one file/database
this applies to xattr use as well.
Very true but it begs the question of generic picture management policy. Posted Jul 4, 2010 18:21 UTC (Sun) by bjartur (subscriber, #67801) [Link]
It's called "filesystem" :P
Ideally, I'd like simply having mountable files (on Linux you'll need a loop device; preferably in user space) that can be formatted as a directory (multiple named internal data streams), list (multiple indexable streams) or whatever may be invented in the future. Simple bencode some data and mount it to be able to work with it The Unix(R) Way. The tools that work with the data need not care how it's stored on disk, so you can just choose the most efficient format or whatever suits you.
Use directories Posted Jul 4, 2010 21:14 UTC (Sun) by bjartur (subscriber, #67801) [Link]
Copying isn't necessary: if moving the physical location of files *requires* a namechange the naming system is broken. It's purpose is to abstract physical location away. If it can't do that, it isn't working properly.
Categorizing images into folders using hiearchial directories should work fine. If you use symlinks or bind mounts you'll get full device abstraction and even just union mounting gets you some abstraction (though it may pose some limits on the storage of labels (i.e. names of folders)). Union mounting may simplify management if the set of folders is quite static.
unfortunately this misdesign is common Posted Jun 17, 2010 13:38 UTC (Thu) by marcH (subscriber, #57642) [Link]
> There seem to be any number of photo applications that want to suck your photos into some internal database somewhere, [...] and commits you to using only that app.
It's called a BKL: Big piKtures Lock.
|
|||||||||||||