Distributed and Offline Wiki Clients

Dear all,

i'm trying to setup an intranet and knowledgebase based on dokuwiki. My plan is to have a star topologie, where all the clients do a synchronisation with a central dokuwiki. Together with a local running apache each client would have an offline dokuwiki which can be synchronised with the central dokuwiki (using unison). Due to the fact that pages are added in the background the search index is not longer functional or complete on both sides. Is there a way in the current production release to build up the index from the commandline, because i want it to be done automatically by a script.

Do you also also have an practical approach how the changes.log could be brought/merged together with no user interaction.

Any concepts or ideas to build a distributed offline wiki are welcome

regards — Stefan Bärthel 2005-12-27 14:55

Would it not be easier to have a central web server, and use proxy servers. I'm not talking about setting a proxy server in the browser, but using a web server that fetches content from a master server. One such procject is Varnish, but you could proably use Squid and other caches

regards — Anonymous hacker

I use a very small scale unison merge between my laptop, my wife's pc and the 'net. I also use unison to sync it with a script to do the diff3 merge and invoke vim for manual merge on failure. Merging the changelog is the easy part

Here is a snippet from the script I use via unison:

merge = Name * -> /home/ant/bin/dokumerge.sh /var/wikidata/caetanowiki CURRENT1 CURRENT2 NEW CURRENTARCHOPT

which does the merge of the changes.log file in the case of their being a common archive copy (ORIG) or not… Hopefully it is understandable. This can also be used via the mergebatch tag since it will always work… unlike the diff3 which will fail if you have both editted the same part of a page.

#
# merge a dokuwiki changes.log file
function merge_changelog
{
        log ">>> merge_changelog"
        if [ -n "$ORIG" -a -f "$ORIG" ]; then
                cp $ORIG $NEW
                # add added (+) lines in LEFT and in RIGHT to the NEW file and sort
                (diff -U 0 $ORIG $LEFT | sed -e '1,3d' | grep '^\+'; diff -U 0 $ORIG $RIGHT | sed -e '1,3d' | grep '^\+') | sed -e 's/^.//' | sort -n >> $NEW
 
                # set timestamp to the latest one
                if [ $LEFT -nt $RIGHT ]; then
                        touch -r $LEFT $NEW
                else
                        touch -r $RIGHT $NEW
                fi
        else
                # add added (+) lines from LEFT to RIGHT to LEFT file and sort
                (cat $LEFT; diff -U 0 $LEFT $RIGHT | sed -e '1,3d' | grep '^\+' | sed -e 's/^.//') | sort -n >> $NEW
        fi
 
        log "<<< merge_changelog"
}

I am not sure how you will maintain the star topology without a lot of work or a strict acl system that prevents people editing on the same file. A diff3 goes a long way but if there is a conflict somebody has to merge it by hand. My script currently starts vim (if in non-batch mode) on the failed diff3 output and I manually finish the merge… You have to be savvy for this (i.e. not a normal user action) and further if you have multiple merges running I am not sure what happens! It would be best then to initiate the merges from the central machine out with -batch to catch most changes, but you will have to manual sit there and work through conflicts. I get annoying conflicts on file properties and modification times (which is why the touch is there, although it doesn't help) and sometimes unison loses its archived copy which is why i change the script to use CURRENTARCHOPT instead of CURRENTARCH…

Have fun!

— Anthony Caetano 2005-12-28 15h-ish

with the concept i'm trying to realize it is not possible to prevent conflicts. I have wrote a script which is used for synchronisation. If unison find a conflict it has to be resolved with currently with kdiff3 or winmerge (im working with windows).

My idea or hope is, that most of the changes are unique. So if some peoples are editing the same page i think it is for adding new content not for changing old content. In the case of a conflict most of them are easy to solve or automatically merged by kdiff3. In my view this is acceptable. Together with the fact, that we are a small group up to 10 peoples, i think it will be very rare that exact the same content will be changed. Of course this is the main problem with this solution.

I played around with unison I found out, that after merging conflicts unison will check the consistency of the files, so if there are multiple merges running it will report if the remote site changed during your synchronisation/merging and you have to synchronize again.

thanks for the input for the changelog problem… to share my windows port you find the attached code below. It is a little bit simplified because i always use the local current changes.log file as base… also the pipelining didn't worked as under unix.

::# Unix tools used in this script: sed.exe, grep.exe, diff.exe and cat.exe
::# 01.01.2006
@echo off
setlocal
 
SET CURRENTPATH=%~dp0%
SET UTILS=%CURRENTPATH%
SET TEMPFILE=%temp%\changes_temp.log
SET LOCAL=%1
SET REMOTE=%2
SET NEW=%3
 
%utils%cat "%LOCAL%" > "%tempfile%" && %UTILS%diff -U 0 "%LOCAL%" "%REMOTE%" | %UTILS%sed -e "1,3d" | %UTILS%grep "^\+" | %UTILS%sed -e s/^.// >> "%tempfile%" && cat "%tempfile%" | sort > "%NEW%"
del "%tempfile%"
endlocal

thanks a lot

— Stefan Bärthel 2006-01-01 17:56

WHO CAN EXPLAIN MORE ABOUT CHANGE_LOG MERGING? I REALLY DON'T UNDERSTAND. THANKS FOR YOUR HELP!!!