LWN: Comments on " Apache Subversion 1.11.0 released"

Hmm …

moltonel — Sat, 17 Nov 2018 11:02:24 +0000

In this case they're not too hard to find, I know I tried them myself to solve the "work with multiple branches in parallel" problem. It's just that they all have drawbacks and in the end, `cp -a` works better for me.

It's worse with the "proper file rename support" problem though: in that case all the experts had to propose were ways to split/group two rename/modify commits together, which are not only horribly complicated, but don't solve the actual problem. The workaround of separating commits into a "pure rename commit" and "the rest of the commit" (whatever the level of automation) is really not satisfying.

Hmm …

aleXXX — Fri, 16 Nov 2018 22:47:16 +0000

Here we see one issue of git in action.
Somebody raises an issue or question, and thi starts a discussion between git experts who all propose different non-obvious git commands.

Hmm …

mathstuf — Wed, 14 Nov 2018 15:28:39 +0000

It's just used during `git rebase` (AFAIK). It stores your local diffs until the rebase is over and then applies them (keeping them in the stash if there are conflicts). It also comes into play when using `git pull --rebase`.

Hmm …

jezuch — Wed, 14 Nov 2018 08:29:49 +0000

There *is* autostash but I haven't used it so I don't know what it does or when :)

Hmm …

arnout — Tue, 13 Nov 2018 22:51:22 +0000

> No, what I mean is when you 'rm' a file or two in the working copy and then you want to restore those files
> from the current commit (or from git's index, if they happen to be in there).

You might want to give 'git checkout -p' a try.

Hmm …

mathstuf — Mon, 12 Nov 2018 16:26:15 +0000

> (FWIW, I've (ab?)used this property of git to great effect in some projects, e.g. tracking an automatically modified version of one tree in another, where you just have to provide a tree->tree mapping "function", and then git will sort out everything else basically by itself)

This is what we do to track third-party library imports in our projects. We take the upstream tree, select out files we want and then `git merge -Xsubtree=import/path/` into the "right place" in the project tree. There are then commit checks to ensure that all changes do that subdirectory are done via the importing process.

Hmm …

mathstuf — Mon, 12 Nov 2018 16:24:05 +0000

I'd say you just give it to `git bisect start`. Offhand, I imagine it could basically treat all non-first-parent commits as `skip`. When it finds out that it is one of the X auto-skipped commits, it unmarks the first-parent history of those candidate commits and then restarts the bisect on those commits.

Hmm …

smurf — Mon, 12 Nov 2018 08:26:26 +0000

> which makes me wonder....

If you only do first-parent then you may end up at the point where your only possible refinement is to start bisecting a group, which you forbade git to do – after all, you used "--first-parent".

Thus, this situation requires additional code – which hasn't been implemented yet.

Hmm …

neilbrown — Mon, 12 Nov 2018 04:17:55 +0000

> I think this would be handled well with a `git bisect --first-parent` option ...

What version of git do you need? Do you give "--first-parent" when you "git bisect start" or on every "git bisect good/bad"??

"revision.c" in current git contains the line

revision.c: die(_("--first-parent is incompatible with --bisect"));

which makes me wonder....

Hmm …

neilbrown — Sun, 11 Nov 2018 21:59:10 +0000

> And how is that going to help us noobs (who don't have a great understanding of git)?

The best way to help noobs like you is through education.
An important part of education is encouraging exploration and experimentation.

If something goes wrong and you try to fix it up and fail, and then post in some community "Hi, X happened so I tried Y and now Z", then someone says "Oh no, never try Y - that is a disaster, I hope you have backups" - then you might not want to experiment again.
If, instead, they could say "Y probably wasn't the best approach, but git has your back - just run this simple command and look through the resulting log until you see the code you lost - then do Q to recover it" - you might be more likely to experiment more next time.

So I agree that auto-git-stash is not a complete solution, but it does seem that it would still be valuable.

Hmm …

jezuch — Sun, 11 Nov 2018 18:37:28 +0000

I agree that names could be better. This is one more illustration of evolution over intelligent design ;) But after a while of using git it's all just... identifiers.

Hmm …

Wol — Sun, 11 Nov 2018 17:58:26 +0000

And how is that going to help us noobs (who don't have a great understanding of git)?

I tend to use git with a bit of a cookbook aide-memoir, I have an understanding of what's going on but it's not particularly good. An automatic "git stash" will be a pretty good way of helping me lose stuff, because I haven't yet got to grips with what stash actually does. Yes I know it puts stuff "somewhere safe", but as it is at home, "somewhere safe" usually equates to "I've forgotten where I put it" ie I've lost it.

Cheers,
Wol

Hmm …

flussence — Fri, 09 Nov 2018 22:07:44 +0000

I once spent some time using git-svn to work with an ancient subversion server (for practical reasons - the git clone was much smaller on disk and `git commit` was about 15 minutes faster than `svn ci`).

That didn't last unfortunately; somehow it led to the svn server writing corrupt metadata to its own DB that it couldn't read, and I ended up spending a day to repair it by hand.

(Maybe this anecdote says more about the working conditions at that gig than SVN, but it was still an unpleasant bug to run into.)

Hmm …

mathstuf — Fri, 09 Nov 2018 19:55:37 +0000

We do this by having a `Backport: <branch>` trailer in the description of the merge request which the merge robot then takes and merges into the required branches. The full syntax is `Backport: branch:committish` where `HEAD` is replaced by the commit at the top of the MR. Need to resolve a conflict? Merge on your MR branch and then do `Backport: release:HEAD^2` to get the release fix into that branch.

Hmm …

mathstuf — Fri, 09 Nov 2018 19:53:39 +0000

One class in college had a shared SVN repo for the whole class. Some enterprising soul managed to commit `.svn` to the repo. How? I have no idea. But the server was decided unhappy with the situation.

Hmm …

mathstuf — Fri, 09 Nov 2018 19:52:20 +0000

> 'git bisect' could become aware of groups (bisecting at top-level granularity first, then optionally breaking down to the next level to find the particular sub-commit

I think this would be handled well with a `git bisect --first-parent` option which bisects the first-parent history and then starts bisecting the history between the two commits recursively on the remaining history using the same first-parent semantics. Then your "groupings" are exactly merges.

Hmm …

Wol — Wed, 07 Nov 2018 13:53:01 +0000

> There is no such thing as a branch in SVN; by convention only, the root of the repository contains three directories, "tags", "branches" and "trunk". You are quite at liberty to check out the root and thus have all branches plus all tags plus trunk in a single working copy, and to commit a single commit across branches, tags and trunk at once.

Problem is, there's no such thing as a root in git. That's the whole point of a DVCS :-) The only reason Linus' tree is the root for linux is that everybody agrees that it is.

Having worked with Visual Sourcesafe, and with git, git is so much simpler because you're not stepping on other peoples' toes all the time :-)

Cheers,
Wol

Hmm …

jezuch — Wed, 07 Nov 2018 11:12:14 +0000

If there is only one line changed then git will almost certainly get this right, unless the file is very small. The heuristic is configurable, also, and the default for the "effort" parameter (don't remember what it's actually called) is too small in my opinion. I always jack it up way, way higher.

AFAICT the git developers will refuse addition of an explicit rename on the grounds that it's an "information tracker" and in theory it could also highlight code moved around in other ways. On the other hand the most recent release will also figure out that an entire directory was renamed, so if you add a file into that directory on one branch and rename the directory on another, when you merge these branches the newly added file will also be moved along with the rest of the files in that directory. It used to drive me nuts when I did merges like this in the past :)

Hmm … commit groups

neilbrown — Tue, 06 Nov 2018 23:02:05 +0000

From a pragmatic perspective (rather than aesthetic) you seem to have identified two particular elements of functionality.

For the moment I'm thinking of a group as a merge commit plus all the commits on the right side of the merge that aren't also on the left (for some reasonable understanding of "left" and "right").

1/ you want "git bisect" to recognise these merge points and preferentially test those, rather than commits on the right. I suspect it would be fairly easy to do this for all merges. It would be expected to increase the number of steps a little, but probably just by one or two.

2/ you want to allow a new commit to mark an existing commit as part of it's group. This is comparatively huge. For git-bisect to be able to notice, you would need to create some sort of index from child commits back to their parents, at least for merged commits.
Between 4.1 and 4.19 (just a random sample) there are 20808 merge commits and 232659 non-merges. That means about 10% of commits are merges. Adding an index for the children of all merges should be manageable. If it was just "all merges with some magic string in the message" it would be even more manageable. I'm not sure if that is quite enough to allow git-bisect to do what we want.

I've occasionally thought this would be good for patches labeled "Fixes:". If they were attached to the patch they fix in a way that git-bisect could detect, then maybe it could always pull in fix - or a least warn that a fix was missing.

I think there are ideas worth exploring here (if only I had the time).

Hmm …

neilbrown — Tue, 06 Nov 2018 20:14:03 +0000

> It's more about 'git checkout .' which will silently overwrite every local change in the working copy.

Anyone want to write a patch which causes "git checkout" (and anything else potentially destructive) to do an automatic "git stash" first ???

Hmm …

epa — Tue, 06 Nov 2018 20:09:38 +0000

Yes, that's the core of my complaint: it sounds a lot more innocuous than it is. If it were called 'git overwrite-local-changes' there wouldn't be any issue. I think that being called 'checkout' it should prompt a bit more before trashing multiple files.

Hmm …

epa — Tue, 06 Nov 2018 20:08:01 +0000

Thanks!

Hmm …

MrWim — Tue, 06 Nov 2018 19:44:40 +0000

Put this in git-group.sh, mark it executable and run git config --global alias.group '!/path/to/git-group.sh'. You will then be able to run git group open and git group close. It's a quick and dirty hack, with precious little error handling, but it will support nesting and should be convenient to use.

If you want to get a log with these groups collapsed run git log --first-parent.

git-group.sh follows:

#!/bin/sh -e

fail() {
    echo $@ >&2
    exit 1
}

current_branch=$(git symbolic-ref --short HEAD)

cmd=$1
shift

case "$cmd" in
open)
    git checkout -b ${current_branch}-group
    ;;

close)
    parent=$(echo $current_branch | sed 's/-group$//')
    [ "$parent" != "$current_branch" ] || fail "Not in a group!"
    git checkout "$parent"
    git merge --no-ff --no-commit --log "$current_branch" --edit
    printf '# Enter top-level commit message here.  Sub-commit messages follow:\n\n' >.group_commit_msg
    git log --first-parent --format=%B "${parent}...${current_branch}" >>.group_commit_msg
    git commit --file .group_commit_msg --edit
    rm .group_commit_msg
    git branch -D "$current_branch"
    ;;

*)
    fail "Unknown command $cmd"
    ;;
esac

Hmm …

jezuch — Tue, 06 Nov 2018 15:55:03 +0000

One small nit: "git checkout <path>" will trash all changes that are not in the index (AKA staging area). I use it all the time to trash only select changes: "git add -p" and then "git checkout ." usually followed by "git reset". It's very powerful. But you need to have total situational awareness of your working copy for this to work without surprises.

Hmm …

smurf — Tue, 06 Nov 2018 13:45:25 +0000

IMHO a group marker shouldn't mark the "previous N commits" to be a group (that won't work if one of these is itself a merge to something "outside" the group) – instead, it should declare a specific ancestor to be the one that's displayed when the group is collapsed.

That special-ness could be even signalled without touching the core data structure: a normal no-follow merge commit mentions its parents in a specific order (the "null" parent is first). A "this is a group" commit would simply invert that.

Hmm …

epa — Tue, 06 Nov 2018 12:00:19 +0000

I suspect that no new low-level machinery would be needed; instead a special type of commit declares the previous N commits (with the given SHAs) to have been part of a group, with a certain log message. 'git bisect' could become aware of groups (bisecting at top-level granularity first, then optionally breaking down to the next level to find the particular sub-commit, and so on) and 'git rebase -i' would need to show them somehow, and do something reasonable when you want to rewrite history changing the contents of a group. Then web interfaces like Github would let you review the grouped commit with an ability to click through to see the sub-commits it contains.

Perhaps all this could be done with tags, I'm not sure -- but I like the idea that groups are immutable once created (just like ordinary commits), yet at the same time you can push a new group-declaring commit retrospectively making some previous commits part of your group. (I think it might be wise to forbid group spaghetti where a commit can be part of two overlapping groups, except for the straightforward case of nesting. That is something a git hook could take care of. The git machinery itself doesn't really get affected by that, however.)

Hmm …

epa — Tue, 06 Nov 2018 07:08:30 +0000

Yes, as I mentioned you can get some of the benefits by merging a branch in one commit. But that forces a nonlinear history, while allowing nested changes would still give a linear timeline for bisecting, etc.

Also it only really gives one level of nesting and is a bit clunky. Take for example the problem mentioned in an earlier post. To rename a file you may first do ‘git mv’ in one commit then change the contents in the next. Does anyone really land a separate branch just for that? What I envisage is something like ‘git group open; git commit; ...; git group close’. On closing the group you are prompted for a top-level commit message. Or you could group commits when rebasing. This would be similar to the current support for squashing commits into one, but still allowing the individual changes to be retrieved if you choose to ‘look inside’.

Hmm …

smurf — Tue, 06 Nov 2018 06:28:29 +0000

Grouping commits is reasonably simple – you create a branch for them, and then do a non-fast-forward merge to the parent.

Hmm …

neilbrown — Tue, 06 Nov 2018 03:06:11 +0000

In what way is it "clunky by comparison"?
Would it require changes in the git engine to achieve what you want, or would it be sufficient to add some commit-group-aware interfaces?

Hmm …

epa — Mon, 05 Nov 2018 19:47:50 +0000

I often miss the ability to group commits in git -- so one top-level commit could be unpacked into smaller ones if you choose to view it that way (and those smaller ones themselves could be nested commits, etc).

That would take care of the 'doing this commit just so git doesn't become confused' problem and it would also help with patch sets and work on a particular feature -- where the whole work is merged as a single logical commit, but by bumping up the default nesting level you can drill down to the component changes. You can emulate this a little bit with branching and merging (where there is a single commit merging the feature branch) but it's clunky by comparison.

Hmm …

johill — Mon, 05 Nov 2018 18:47:07 +0000

> Svn operates in terms of tree snapshots while git operates in terms of changesets.

Hm. I think that came out wrong?

git doesn't really care about the changes at all, each commit just records the current state of the tree and the parent commit.

Perhaps you meant to say something else?

(FWIW, I've (ab?)used this property of git to great effect in some projects, e.g. tracking an automatically modified version of one tree in another, where you just have to provide a tree->tree mapping "function", and then git will sort out everything else basically by itself)

Hmm …

ms-tg — Mon, 05 Nov 2018 17:33:58 +0000

Yes, I was just about to type the same reply. The git mv heuristic's most inconvenient failure case is the one you mentioned -- when renaming a source file also implies that you must change the naming inside the files. Considering how very common this really is across widely used programming languages, I'm also surprised that this is still an issue today.

Instead of doing what seems semantically correct, we instead must:
1. Locally git commit nothing more than the 'git mv OLDNAME NEW NAME'
2. Locally git commit the naming change inside the file and all referring files

And note in the commit history that commit (1) doesn't actually work, but is kept separate so that git can identify the rename in the history.

Does anyone have a better standard practice for the above today?

Hmm …

NAR — Mon, 05 Nov 2018 13:53:28 +0000

For some languages the filename and the "module/class name" in the file must be the same, otherwise the code doesn't even compile. Only renaming a file, but not changing it leads to broken build - so if we want all commits to be able to be built, the file has to be modified at the same time as it's being renamed and sometimes git doesn't notice that it's a file rename :-(

Hmm …

anselm — Mon, 05 Nov 2018 12:29:30 +0000

An svn commit is actually better at "completely and unambiguously describ[ing] one particular state of your work (and its history)" than a git commit, because it encompasses all the repo's branches, not just the branch(es) the commit belongs to.

That depends on your philosophy. Svn operates in terms of tree snapshots while git operates in terms of changesets. With git, it is straightforward to see which sequence of changesets has resulted in the current state (“branch”), and it could be argued that the exact state and provenance of other branches actually isn't that important at that point. Svn gives you the “big picture” and of course that may also have its advantages every so often. OTOH, in git, a “branch” is a much more lightweight object than in svn, and that encourages the use of single-feature branches where you end up with a whole lot of them.

Git also makes it easier to transplant changesets from one branch to another after the fact, or to publish or consume them remotely, and that of course is at the root of the whole “decentralised” thing that svn doesn't really address at all. Working with git, I find myself doing rebases and fixups quite a lot just to keep the development history of my code nice and logical, and unless svn has improved considerably in that area since I used it I have no idea at all how I would do that with svn.

Hmm …

farnz — Mon, 05 Nov 2018 08:18:32 +0000

Sure, people don't normally work that way - but in special cases, you can do that if you have to.

And I'm surprised that you think that ordinary customers on the phone have the test suites, a full ancestry tree and the ability to query that tree. That's not my experience anywhere - and in the sort of place where you fix all branches at once, you probably also have customers with special case branches with a per-customer fix on it. In that case, a BUGS_FIXED file isn't helpful - it'll cover all the cases you've thought about, but not the special cases.

Also, this isn't about whether or not this is good practice (small shops do all sorts of things that aren't good practice!) - it's whether or not there's anything that git can't do that SVN can. This is something that SVN can do, partly by virtue of not being as complete a VCS, that git can't do.

Hmm …

epa — Mon, 05 Nov 2018 06:59:18 +0000

No, what I mean is when you 'rm' a file or two in the working copy and then you want to restore those files from the current commit (or from git's index, if they happen to be in there).

'git checkout .' will appear to work for this task but it is highly dangerous, since it unconditionally trashes any other local changes. As another poster noted, there doesn't seem to be a command which does it, short of running 'git status', looking for 'deleted' lines, and running 'git checkout' over those.

Now, you may say that this isn't a sensible way to work, and the right way to revert changes in a file if you messed it up is to run 'git checkout FILE' directly. Not to just delete the file and then hunt for the right incantation to bring it back. But rightly or wrongly it is a natural human reaction to say "I really messed this up, let's just delete it and start again".

I did raise the feature request on the mailing list: https://public-inbox.org/git/loom.20150603T104534-909@pos...

Hmm …

smurf — Sun, 04 Nov 2018 20:36:11 +0000

Sure I can check out everything at once. But (a) people usually don't work that way, (b) that also means that I can break everything at once. Guess what I, wearing my management hat, don't want to enable people to do – there's a reason we now have CI systems that run mandatory test suites before allowing anybody to check their "bug fix" into the release or CD branch.

Yes, you need an ancestry tree (not a full one – just one that goes back to the parent of the bugfix) if you do this with git. Surprise: You are most likely to already have it. In any case, there are other ways; one no-brainer method is an entry in a BUGS_FIXED file you'd check. Another would be a test suite that actually verifies that the bug is (and stays) gone.

Hmm …

madscientist — Sun, 04 Nov 2018 20:11:05 +0000

Note the OP includes a "." at the end of the git checkout command:

> It's more about 'git checkout .' which will silently overwrite every local change in the working copy.

Adding a "." causes a checkout of all (modified) files from the current directory down. It cannot go to a different commit.

For me, if I want to un-modify a file I use "git status" to see a list of modified files, and "git checkout filex filey ..." (perhaps copy/paste) to check out specific files. If I want to throw away everything I use "git reset --hard HEAD" or similar. I virtually never use "git checkout <directory>" and wouldn't suggest it to others, either.

> what is the safe git command to bring back deleted files

By this I assume you mean a committed file that you have deleted and then want to recover and not an untracked file that you've deleted.

The best way to know how to solve almost ANY issue in a workspace is to run "git status" and it will tell you how to undo it. For example, if you "rm somefile" and then decide you want it back, "git status" will tell you:

> Changes not staged for commit:
> (use "git add/rm <file>..." to update what will be committed)
> (use "git checkout -- <file>..." to discard changes in working directory)

> deleted: Demo/readme.txt

If you use "git rm somefile" then decide you want it back, "git status" will tell you:

> Changes to be committed:
> (use "git reset HEAD <file>..." to unstage)

> deleted: Demo/readme.txt

And finally when in doubt, I use "git stash" to stash modified files before doing any kind of bizarre operations on a repository where there are modifications I want to preserve, just in case.

Hmm …

mpr22 — Sun, 04 Nov 2018 20:05:57 +0000

How often does the heuristic get it wrong if you don't make life impossible for it by making wholesale changes to the file in the same commit that you rename it?

Hmm …

smurf — Sun, 04 Nov 2018 19:03:52 +0000

> It's more about 'git checkout .' which will silently overwrite every local change in the working copy.

That depends on whether you want to get a single file (or three), or go to a different commit. The latter refuses to overwrite things.

> what is the safe git command to bring back deleted files

There is none. If you overwrite a file that hasn't been checked in, it's gone, just like when you use "rm". In fact, if you try to "git rm" a modified file, it'll warn you.

Adding a warning option does make sense; I'll file a feature request.