[Adium-devl] xml format possiblities and drawbacks

[Alan Humpherys]

Well Said Graham,

If I can put in my 2 cents worth...  I too was sceptical as we made  
the transition in fire to use XML (and XSLT) for our message format  
to go to the display and logs.  My main concerns were performance  
related.

by doing what Graham discussed, we don't have performance issues, and  
the benefits are good:

1) The log files can be viewed in any web browser - and have a decent  
rendering, even without an XSLT transformation.

2) Searching via Spotlight as well as our internal log browser was  
able to get the conversation contents without having troubles with tags.

3) All aspects of the conversation were preserved.  Including  
formatting, timestamps, signing, and encryption flags.  They could be  
viewed later without any loss of content.

We still have an occasional person who wants to export them as a text  
file, or use grep, but I think that it is a valid response to tell  
people to use spotlight and not grep.  Also, the export can be  
accomplished by opening the file in a web browser, and using cut and  
paste.

We went the route of having each conversation separated in its own  
file, (conversations have a demarcation of opening and closing of the  
conversation window in the UI).  The conversations with each user  
were separated into their own directory, so that someone looking for  
a conversation with a particular person, could actually find it  
fairly easily by looking in the directory structure, for a particular  
SN with a datestamp for the filename of the actual conversation.

The final answer.... I think well designed XML is the way to go for  
the logs.

Alan
______
Alan Humpherys
alan at ewonderz.com



On May 23, 2006, at 9:31 PM, Graham Booker wrote:

> Speaking from experience here (I did the XML logging in Fire after  
> reading the code in Colloquy):
>
> 1) Solved by incremental updates to the XML log file.  We already  
> have a solution for this in place in Adium through the  
> AIXMLAppender class.  This doesn't require reading the entire log  
> from disk, and writing the entire new log to disk for each message,  
> unlike what Kopete was doing.  Fire uses incremental updates and I  
> can conclusively say that I have never noticed a single slowdown.   
> Colloquy is the same, and I have log files there in the several  
> megabyte range.
>
> 2) I think the main gripe here is viewing the file outside the  
> app.  I would recommend a web browser for this.  In Fire, we didn't  
> have a single gripe about the XML log format over the simpler text  
> based format.  Also, conversion to text is a simple operation, and  
> grep still works on the XML files.
>
> 3) The main gripe here (about XML) is that the history used a  
> different XML format than the internal XML format that was fed  
> through the XSLT for display.  This meant that there were  
> essentially two XML formats to deal with, one for the display, and  
> one for the history.  The solution is to use the same format for  
> both, which is what we did in Fire (and so does Colloquy).  Adium  
> doesn't currently use an XML format to feed through an XSLT for  
> display, so this doesn't apply.  If it were to in the future, using  
> the logging format is a natural choice unless there is an  
> exceptional reason to not use it.
>
> 4) The gripe here is search speed.  We get around much of that  
> through Spotlight, but for those without spotlight, it is quite  
> trivial to search for content very quickly in XML.  Simply toss all  
> the tags and fix the XML entities.  This will give you the plain  
> text of the log to search, and is a much cheaper operation than XML  
> parsing.  The is the mode we used in Fire, and the slowdown for a  
> linear search through files (non-spotlight) was slightly slower,  
> but not noticeable over parsing an HTML log (since you have to do  
> the exact same thing there too).
>
> Hope this helps put some fears to rest.
>
> - Graham
>
>
> On May 23, 2006, at 9:54 PM, Matt Rogers wrote:
>
>> Hi all,
>>
>> I heard from one of the gaim guys that y'all were working on a new  
>> XML format
>> for the logs. I was talking with Catfish_Man about some of the  
>> problems
>> Kopete has had with the use of XML in the log files. Here's 4 bugs  
>> that I
>> found that express some of the difficulties users have had with  
>> the log file
>> format and it's resulting implementation in Kopete. I'm sure we  
>> can do it
>> better than we are, so this is just food for though for the  
>> discussion.
>>
>> http://bugs.kde.org/show_bug.cgi?id=65982 - updating of chat  
>> window takes ages
>> when there are many messages
>>
>> http://bugs.kde.org/show_bug.cgi?id=89744 - request to change  
>> history format
>> to plain text for easy searching from the command line
>>
>> http://bugs.kde.org/show_bug.cgi?id=94178 - Kopete's chat log  
>> system is
>> confusing (talking about on-disk layout mostly)
>>
>> http://bugs.kde.org/show_bug.cgi?id=113794 - History viewer is slow
>>
>> Again, this is just food for though for the discussion. Learning  
>> from our
>> mistakes and all that. :)
>> --
>> Matt
>> (Please CC any replies, i'm not subscribed)
>>
>> _______________________________________________
>> Adium-devl mailing list
>> Adium-devl at adiumx.com
>> http://adiumx.com/mailman/listinfo/adium-devl_adiumx.com
>>
>
> _______________________________________________
> Adium-devl mailing list
> Adium-devl at adiumx.com
> http://adiumx.com/mailman/listinfo/adium-devl_adiumx.com