Ticket #1035 (closed enhancement: fixed)

Curtis reports that the performance improved with switching the Git plugin's caching *off*.

The major problem now seems that we need a way to somehow deal with the many references to Subversion revisions. Either we convert them to hashes (which would be a one-time, clean break) or we hack something like "r<n> -> 79599fe5f~<4992 minus n>".

My vote is for the one-time conversion. Trac now allows editing of both description and individual comments, so it should be doable to write a script that updates all "r\d+" patterns to their corresponding Git hash. Then we don't have to hack the commit hashes themselves, and the Trac links will not be broken.

One thing we probably want to do is to activate the Trac plugin on  https://github.com/imagej/imagej/admin/hooks to update the ticket statuses automatically.

We used the script bin/rewrite-svn-branch.perl to rewrite all of our branches and replaced 'master' with that history in in 2defeef21c2c2dc5c96a79832e01adc94d8466ce. The conversion script was subsequently removed (but it is still part of history).

To rewrite the tickets in Trac, we used this script:

#!/bin/sh

PERL=/tmp/trac-svn2git.$$.perl
TRAC=/var/lib/trac/imagej/db/trac.db

cat > $PERL << EOF
our %mapping = (
EOF

(cd $HOME/ij2 &&
        git log master |
        perl -e '
                my $commit = "";
                my $sep = "";
                while (<>) {
                        if (/^commit ([0-9a-f]{40})/) {
                                $commit = $1;
                        }
                        elsif (/^    This used to be revision (r[1-9][0-9]*)\.$/) {
                                print $sep . "\"" . $1 . "\" => \"" . $commit . "\"";
                                $sep = ",\n";
                        }
                }
                print "\n";
        ') >> $PERL

cat >> $PERL << \EOF
);

sub map_revision ($) {
        my $text = $_[0];
        my @list = split(/\b(r[1-9][0-9]*)\b/s, $text);
        for (my $i = 1; $i <= $#list; $i += 2) {
                if (exists($mapping{$list[$i]})) {
                        $list[$i] = $mapping{$list[$i]};
                }
        }
        return join('', @list);
}

while (<>) {
        while ((tr/'// % 2) == 1) {
                $_ .= <>;
        }
        if (/^INSERT INTO "ticket" VALUES\((\d+),(NULL|'([^']*)'),(\d+),(\d+),(NULL|'([^']*)'),(NULL|'([^']*)'),(NULL|'([^']*)'),(NULL|'([^']*)'),(NULL|'([^']*)'),(NULL|'([^']*)'),(NULL|'([^']*)'),(NULL|'([^']*)'),(NULL|'([^']*)'),(NULL|'([^']*)'),(NULL|'(([^']|'')*)'),(NULL|'(([^']|'')*)'),(NULL|'([^']*)')\);$/s) {
                my $id = $1;
                my $type = $2;
                my $time = $4;
                my $changetime = $5;
                my $component = $6;
                my $severity = $8;
                my $priority = $10;
                my $owner = $12;
                my $reporter = $14;
                my $cc = $16;
                my $version = $18;
                my $milestone = $20;
                my $status = $22;
                my $resolution = $24;
                my $summary = $26;
                my $description = $29;
                my $keywords = $33;

                my $new_summary = map_revision($summary);
                if ($new_summary ne $summary) {
                        print "UPDATE ticket SET summary=" . $new_summary . " WHERE id=" . $id . ";\n";
                }
                my $new_description = map_revision($description);
                if ($new_description ne $description) {
                        print "UPDATE ticket SET description=" . $new_description . " WHERE id=" . $id . ";\n";
                }
        }
        elsif (/^INSERT INTO "ticket_change" VALUES\((\d+),(\d+),(NULL|'([^']*)'),(NULL|'([^']*)'),(NULL|'(([^']|'')*)'),(NULL|'(([^']|'')*)')\);$/s) {
                my $ticket = $1;
                my $time = $2;
                my $author = $3;
                my $field = $5;
                my $oldvalue = $7;
                my $newvalue = $10;

                my $new_oldvalue = map_revision($oldvalue);
                if ($new_oldvalue ne $oldvalue) {
                        print "UPDATE ticket_change SET oldvalue=" . $new_oldvalue . " WHERE ticket=" . $ticket . " AND time=" . $time . " AND field=" . $field . ";\n";
                }
                my $new_newvalue = map_revision($newvalue);
                if ($new_newvalue ne $newvalue) {
                        print "UPDATE ticket_change SET newvalue=" . $new_newvalue . " WHERE ticket=" . $ticket . " AND time=" . $time . " AND field=" . $field . ";\n";
                }
        }
}
EOF

commands="$(printf  '.dump ticket\n.dump ticket_change\n' |
sqlite3 "$TRAC" |
perl $PERL)"

echo "$commands" |
sqlite3 "$TRAC"

In short, this script looks for the mappings between Subversion and Git revisions (during the rewrite, the git-svn id line would be replaced by something parse-able like "This used to be revision ....") and then scans the summary and description fields in the ticket table and the oldvalue and newvalue fields in the ticket_change table for rewritten Subversion tickets (it leaves those alone for which we do not have Git revisions; this happened e.g. for loci-java commits), writes appropriate UPDATE statements and finally feeds those to SQLite3.