LWN.net Logo

Response from Shotwell team founder

Response from Shotwell team founder

Posted Jun 17, 2010 22:49 UTC (Thu) by adamdingle (guest, #67751)
Parent article: A quick grumpy review of Shotwell

Grumpy Editor,

I'm the founder of Yorba (makers of Shotwell) and have been deeply involved in Shotwell's design from day one. Thanks a lot for trying out Shotwell and for the candid and thorough review. We always appreciate direct feedback and are happy to hear about what people do and do not like.

Grumpy Editor, your review touched on various points about Shotwell, but of course your biggest gripe was about the way Shotwell imports photos and about the relationship between the Shotwell photo library and the file system. I'll first provide brief feedback about some of the smaller points:

- You mentioned the lack of support for RAW photos and zooming into photos. Both of these features have been impemented in trunk and will be present in the Shotwell 0.6 release (coming in the next two weeks).

- As someone else pointed out in another comment, it actually is possible to split an event apart in Shotwell: simply select the photos you'd like to split into a separate event, then choose Events->New Event.

- It actually is possible to remove a tag from several photos at once. Simply select the tag in the sidebar, then select the photos you'd like to remove it from and choose Tags->Remove Tags From Photos.

- We do hope to implement hierarchical tags at some point (that's http://trac.yorba.org/ticket/1401, if you're curious).

OK, and now on to your Major Concern. As you pointed out in your review, Shotwell expects you to import all photos into its library. As you do so, Shotwell indexes their metadata so you can search them in various ways. You can edit photos in Shotwell, but normally your changes stay in the Shotwell universe. Changes get written to files only when you export (or, in Shotwell 0.6, when you use Shotwell's Open in External Editor command, which writes pending changes to a file and then opens an external program such as GIMP on it).

The Shotwell model is common among commercial photo programs; iPhoto, Aperture and (to some degree) Picasa all work this way, for instance. But clearly you want a photo program to use a different model. You'd like to be able to browse through photos easily according to their on-disk file hierarchy, and presumably you'd like photo edits and metadata to be stored in the photo files themselves, at all times. I imagine you'd also like to be able to search for all photos matching a given tag or in a given date range. For this to be possible, a photo program must of course index the on-disk photos, which is itself a form of importing, really. But it sounds like you're opposed to an explicit import step. (Perhaps you'd like to point a photo program at your top-level photo directory and let it index everything in the background. Or maybe you'd like a photo program to automatically index all the photos in directories which the user browses through.)

So, then, which model is superior? We built Shotwell to use explicit importing and to store edits in its own database both because other popular programs work that way and because certain users have told us they want us never to touch their originals (see, for example, the discussion at http://lists.yorba.org/pipermail/shotwell/2010-April/0002...). But recently we've been getting more and more feedback from people like you who want a storage model based directly on the filesystem (see, for example, http://ubuntuforums.org/showthread.php?p=9475456 or https://bugs.launchpad.net/ubuntu/+source/shotwell/+bug/5...). So I've actually become less and less sure that we've gotten this right, and I think it's likely that we'll switch to something like your preferred storage model in a not-too-distant release, at least as an option for the user. In fact, different users seem to have distinctly different storage preferences, and so perhaps we can even evolve Shotwell to the point where the user can choose where data is stored. For example, a user might want us to store metadata (e.g. photo titles/keywords) either in the original files, in sidecar XMP files, in the Shotwell database, or in Tracker. It might be cool if Shotwell could do any of these as you please.

As you mentioned, Shotwell is a young utility. We do in fact plan to evolve it to work more flexibly together with other Linux tools, so thanks again for the feedback and please keep your eyes open for future Shotwell developments!

Cheers,
adam

===
Adam Dingle
Yorba


(Log in to post comments)

Response from Shotwell team founder

Posted Jun 17, 2010 23:07 UTC (Thu) by dlang (✭ supporter ✭, #313) [Link]

I think that the key thing is that the current model is based on the premise that the user will only use one application to touch their pictures.

the Grumpy Editor (and many of the rest of us) want to be able to use multiple different apps to deal with the pictures.

we may use Shotwell to view and organize the pictures, but gimp to edit them, (or imagemagic to resize them, or dcraw to convert from raw, etc)

we could let Shotwell import them (and copy them) into it's own structure, and then point these tools at the results, but this will fail as soon as any other tool also wants to take complete control of the pictures, so Shotwell is viewed as being anti-social by doing so.

We understand that this is common practice for the propriatary software world (windows and MAC especially), but the unix philosophy is to allow you to use multiple tools, allowing the user to pick best-of-breed solutions for each task. It's nice if there is one GUI that can leverage the specialized work and be a single panel for doing the common work, but if that prevents other tools from being used, that GUI can easily become more of a liability than an asset.

It's not strictly necessary for all the metadata to be stored in the initial files, but it should be stored in some way that makes it possible for other apps (and/or scripts) to find, access, and ideally modify this data. putting it in exif is one way (and probably the easiest cross-application method), but not the only way.

This also isn't saying that copies of the pictures shouldn't be made. In fact if you modify the image (especially if you modify the image in a way that cannot be perfectly undone) you _should_ make a copy of the image in some manner before you do so.

Response from Shotwell team founder

Posted Jun 18, 2010 0:32 UTC (Fri) by jspaleta (subscriber, #50639) [Link]

If they implement an edit in another application feature does this solve the problem of multi-application editting?

I think the key question for me is..
what's is the best way to implement a revision control system for images that works well across multiple applications? I really don't want to have to keep oodles of copies of my photos around when I touch them up for different formats. I want revision control in a _smart_ way.

Shotwell's database is just a mechanism to do for images what git does for source code isn't it?

The problem is, we are building dedicated photo revisioning systems into applications silos. What we need is a common revisioning system that _ALL_ imaging processing applications can use. Even commandline usage..where you checkout specific imagine revision, edit the image,then import that back into the image revision database as a changeset (with tags or other metadata as deemed appropriate)

-jef

Response from Shotwell team founder

Posted Jun 18, 2010 0:42 UTC (Fri) by dlang (✭ supporter ✭, #313) [Link]

no, shotwell is like if git was integrated into gedit and you could only access the files through gedit.

git doesn't force you to use a specific tool to manipulate the working copy of the files stored in it.

if shotwell organized the images and left them alone in the filesystem for you to use whatever tool to tweak them, you would not see people raising this objection.

Response from Shotwell team founder

Posted Jun 18, 2010 13:21 UTC (Fri) by paulj (subscriber, #341) [Link]

Insightful comment there.

It follows from what you say that an equivalent model with Shotwell would be if the photos on disk were with modifications applied, with Shotwell keeping data somewhere on how to undo changes (or conversely how to change the original to the current version on disk). ?

Response from Shotwell team founder

Posted Jun 18, 2010 14:51 UTC (Fri) by adamdingle (guest, #67751) [Link]

I agree: it would make sense for Shotwell to store photos on disk with modifications applied, but with the original photo backed up under a separate filename and also storing information about how to change the original to the current version on disk. As some people have observed, it would even be nice if the transformation information could be shared among programs, not specific to one program such as Shotwell. Perhaps we can get there one day. We're interested in modifying Shotwell to use a GEGL pipeline and that might be one step toward a transformation structure that can be shared.

Response from Shotwell team founder

Posted Jun 18, 2010 18:52 UTC (Fri) by dlang (✭ supporter ✭, #313) [Link]

that would be good. I don't think it's that necessary for the exact details of how to convert one to another be available cross application (other than as comments/documentation trail/changelog)

but the unmodified original should always be available (or recoverable, with with lossy compression algorithms means a copy), and the currently 'final' result should be accessable without a need to run some export process.

Response from Shotwell team founder

Posted Jun 19, 2010 2:42 UTC (Sat) by spitzak (guest, #4593) [Link]

Editing should just move the original from X/foo.jpg to X/originals/foo.jpg, and it should throw it away if that file is already there (thus you only keep the most recent edit and the original). If other applications followed this rule as well then you could use any editor and still recover the original.

I would very much like to see a photo manager that just leaves the files where they were. You would give it a list of directories and it will act like the set of photos is every photo found in those directorys and any subdirectories. If a file is in a given directory it acts like it has a tag that is the directory name, as well as any tags that are in the EXIF data. Attempting to remove these "directory tags" would, I guess, move the files, perhaps with a warning. Or add a "not actually tagged with this" tag.

For Linux apps this almost is necessary, just so you can cooperate with some of those proprietary apps that insist on moving the files. For instance I would like to use it to at least look at an rsync copy of a iPhoto directory. If we are careful we should be able to at least change the tags and perhaps edit files and those apps will still handle it.

There may be an "import" but it would be for reading off of cameras and SD cards and other removable media, where you have a good reason to want a copy. The user should be able to choose what directory it goes into, and there should be programmable scripts (with some pre-supplied ones) to use dates and other EXIF data to place the files into subdirectories.

Response from Shotwell team founder

Posted Jun 19, 2010 2:22 UTC (Sat) by Velmont (guest, #46433) [Link]

Have you looked carefully at digiKam?

I've been following Shotwell, because I'd really like to get a less-fat image organizer that digiKam. What digiKam does well:

* You can tag pictures, and captions, and everything is stored INSIDE the picture (I've been burned by this before, using Gallery to write captions for thousands of images and then, all of them lost).
* It's FAST (that's important, I've got thousands of pictures, I want to zoom through them and delete, delete, delete (or watch, watch, watch))
* It doesn't copy or change the pictures
* It does FAST searching (how? well it has it's own indexed database ALONGSIDE the pictures, it's basically just copied the information that already exists inside the images, but done so because it needs to search it).

You can use Tracker to do the searching and indexing-part. I've heard it's much better these days.

The other thing digiKam doesn't do is having an easy way to tag pictures fast (you have to click much with the mouse!).

Anyway, thinking it's way cool you're considering changing the way the software works! :D

Response from Shotwell team founder

Posted Jun 21, 2010 20:49 UTC (Mon) by nix (subscriber, #2304) [Link]

The digikam tags are just stored in a database, so if you shut it down first (or are using a non-sqlite database in recent versions) you can add tags in bulk using raw SQL, restart digikam and write the lot out to the images with a couple of mouse clicks. :)

(but, to be honest, a lot of the time tagging depends on the contents of the images, and for that a bunch of select-mouse-clicks and tag clicks are actually a *good* user interface. Using WIMP GUIs for things that actually are graphical in nature seems like a good idea to me.)

Response from Shotwell team founder

Posted Jun 23, 2010 5:17 UTC (Wed) by Velmont (guest, #46433) [Link]

Well, ... I've got so many tags. I tag with e.g. names, and so I need to write, I'm writing Jack Smi[...word complete...]. Etc.

So what I want is to go to next picture, add tag «jack smi...», add tag «ha» [ppy], next picture. etc.

You can't do that quickly by the mouse. By all means, the mouse ui can stay, I would probably use that for some things, but not for my main tagging.

Response from Shotwell team founder

Posted Jan 6, 2011 22:06 UTC (Thu) by ceplm (subscriber, #41334) [Link]

It is probably too late for any participant of this thread to actually care, but if anybody finds this via Google, most of the "I just want to organize my photos and then process them with something else" camp can be very well pleased with jbrout (http://jbrout.manatlan.com/, packaged for most Linux distros, but check for the latest version). It stores all tags and info INSIDE of the image file (yes, it has cache for fast search, but it is not the authoritative storage of information). Also, it's trivial to have any external application on right-click-menu. Highly suggested.

Matěj

Response from Shotwell team founder

Posted Nov 28, 2011 11:59 UTC (Mon) by Velmont (guest, #46433) [Link]

I did search for my own name to find my user ID on lwn, and lo and behold, a new answer. I'll look at it, looks rather nice.

Would be interesting to see how fast it is. DigiKam is still, one of the fastest image browsers I've come across.

DigiKam loads incrementally better versions of the image, and I believe it decodes the next picture when you have extra time as well. Maybe even it uses a free core on your CPU to do that, I don't know, but I do know that it's miles faster than most other software when looking quickly through huge pictures.

Although I'm no fan of the copy on the website: It's noisy!! Very much so! ;-)

Response from Shotwell team founder

Posted Jun 20, 2010 15:29 UTC (Sun) by mebourne (subscriber, #50785) [Link]

Since you seem to be interested in different use-cases and workflows and since gthumb has just been ruined(*) in the latest version in Fedora 13 (luckily a downgrade to the Fedora 12 package still works fine at the moment) and since there seem to be a preponderance of photo programs like the current shotwell which go against the unix philosophy here's how I manage my 10k's of photos.

I use a script to pull the photos off whichever camera and into the fs, fixing timezone, rotation etc. Photos are placed in a careful directory hierachy organised by event and renamed to MMDD-HHMM to ensure time ordering across multiple cameras, essential at many events (such as weddings). Some photos (such as random shots of the kids around the house) are instead categorised by year and content rather than event.

All photos get checked into a VCS (I use SVN at the moment, though could be anything).

I need a "photo organiser" application that works with my directory hierachy, photos are going to just appear in there from scripts, I don't want any manual import step, I don't want any important metadata stored outside of the photo (and certainly not outside of the directory, gthumb adds a .comments dir which is ok). Storing a cache of metadata for searching would be fine as long as indexing was transparent. It's good to rewrite the image any time I touch it (metadata, or image editing, mostly using gimp anyway) and I don't want the photo organiser to try and do any backup copies or version control itself. After every bunch of edits I commit and if I don't like the changes I can revert.

I check out several copies of the images to different places for different use cases, some full copies and some selected by tags (gthumb's comments are easy to parse from script being in "text" ie. xml).

My main use case of the photo app is rapidly zapping through pictures using the keyboard. Tagging, commenting individually or in groups, deleting (though of course they stay in vcs history). I need a thumbnail organiser but most of the screen needs to be showing the photo and I need keys to zoom into pixel and out again quickly, and full screen/windowed. I might take 10-20 shots of a photo to get a good one and need to see pixel level focus etc, and I need to be able to navigate around them for comparison quickly using the keyboard.

A nice touch would be integration with vcs to be able to view historic commits of a photo for comparison, or to commit changes directly from the app, though the command line works just fine for now. Colour profile understanding would be great too, though colour correcting the whole X display works well enough for now.

(*) The UI has been messed up to make it unusable for my most basic use case and the thing crashes incessantly. Guess I'll just be keeping a fork of the old gthumb which was 9/10 for my workflow until something better comes along. If that is going to be shotwell then great. :)

Copyright © 2013, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds