[mew-int 2494] Re: A patch: make-string in emacs 23

[Harald Hanche-Olsen]

+ Kazu Yamamoto (山本和彦) <kazu at example.com>:

> I'm now sure why Mew tries to aset non-ASCII strings with non-ASCII
> characters. Would you explain the situation?

The problem bit me when a malformed spam ended up classified as
"unsure" by my spamfilter; then mew hit the bug while I was trying to
refresh the folder from the imap server (I use dovecot, in case that
is relevant).

I have probably thrown away the offending spam, but I remembered that
it had uncoded binary data in the From: field, so I managed to
recreate a message that triggers the bug. I attach it, coded as
Application/Octet-Stream. If I drop it into a mail folder where
imapd will find it and point mew at the folder and hit "s", the bug
happens.

The point of the binary data is this: It contains a byte #xf8, which
encodes the character "ø" in Latin-1; then after a space, it contains
the two bytes #xce #xa9, which together encode the greek letter Ω in
utf-8.

The funny thing is this: From my investigations, it seems that text
from the imap server really gets decoded into "ø Ω", which in turn
leads to the problem we're discussing here: Mew creates a unibyte
string, stuffs the letter "ø" into it, then the space, and finally the
"Ω" – which fails, since the string is now a unibyte non-ASCII string.
There seems to be another underlying bug here, in that data from the
server is decoded using what seems like utf-8 with a fallback to
latin-1 for stuff that can't be utf-8?

I'd show you the backtrace, but unfortunately, I had exited the
backtrace buffer, and now mew complains that a process is already
running if I try to redo the steps. And C-c C-k pops me right into the
debugger again!

So here is the backtrace for that error instead:

Debugger entered--Lisp error: (args-out-of-range 1 1497)
  mew-buffer-substring(1 1497)
  mew-insert-buffer-substring(#<buffer %test> 1 1497)
  mew-summary-folder-cache-save()
  mew-imap-sentinel(#<process mew-imap-info-localhost/test> "killed by the user")
  mew-summary-kill-subprocess(nil)
  call-interactively(mew-summary-kill-subprocess nil nil)

Anyway. I just opened a new emacs and triggered the bug there. The
resulting backtrace has very long lines and contains binary cruft, so
I made that into another Application/Octet-Stream attachment. Enjoy,
though it is probably better if you can reproduce the error yourself
using the malformed message.

- Harald
-------------- next part --------------
A non-text attachment was scrubbed...
Name: malformed-message
Type: application/octet-stream
Size: 697 bytes
Desc: not available
URL: <http://www.mew.org/pipermail/mew-int/attachments/20080514/d623c129/attachment.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: mew-backtrace
Type: application/octet-stream
Size: 4183 bytes
Desc: not available
URL: <http://www.mew.org/pipermail/mew-int/attachments/20080514/d623c129/attachment-0001.obj>