segfault in ftp

classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

segfault in ftp

Juan Francisco Cantero Hurtado
This afternoon I was downloading a tarball from a OpenBSD mirror. I
press the key "º" and after the tab key. ftp crashed with a segfault.

Steps for reproduce:
# ftp ftp.fr.openbsd.org
user and password
ascii art
ftp> cd pub/Openº    <- Here press the tab key
segmentation fault (core dumped)  ftp ftp.fr.openbsd.org

It also crashes with the letter "ñ" and "á".

Tested in:
- A snapshot from yesterday. i386. root account. console/ksh without
  locales.
- A snapshot from a few days ago. amd64. user. urxvt/zsh with utf8
  locales.

I also tested the bug in a remote session with OpenBSD 4.7 and ftp works
without problems.

I've updated the code of usr.bin/ftp to 2012-10-01 and 2012-01-01 and
tried both versions. ftp also crashes.

Backtrace:
Thread 1 (process 3436):
#0  memcpy (dst0=0x9d4160, src0=Variable "src0" is not available.
) at /usr/src/lib/libc/string/bcopy.c:115
#1  0x000000000040432b in complete (el=Variable "el" is not available.
) at /usr/src/usr.bin/ftp/complete.c:313
#2  0x000000000041eb84 in el_wgets (el=0x20da64800, nread=0x7f7ffffe3ebc) at read.c:612
#3  0x000000000041ef8d in el_gets (el=0x20da64800, nread=Variable "nread" is not available.
) at eln.c:78
#4  0x000000000040e55f in cmdscanner (top=Variable "top" is not available.
) at /usr/src/usr.bin/ftp/main.c:465
#5  0x000000000040eb7c in main (argc=1, argv=0x7f7ffffe4398) at /usr/src/usr.bin/ftp/main.c:369

Let me know if it's necessary more info or whatever :)

Cheers.

--
Juan Francisco Cantero Hurtado http://juanfra.info

Reply | Threaded
Open this post in threaded view
|

Re: segfault in ftp

Juan Francisco Cantero Hurtado
Chris Cappuccio sent me a mail saying he can't see the characters, only
a question mark.

I'm linking each character to their wikipedia page, so you can
copy-paste the character.

On Thu, Oct 25, 2012 at 05:07:34AM +0200, Juan Francisco Cantero Hurtado wrote:
> This afternoon I was downloading a tarball from a OpenBSD mirror. I
> press the key "?" and after the tab key. ftp crashed with a segfault.
https://en.wikipedia.org/wiki/%C2%BA
>
> Steps for reproduce:
> # ftp ftp.fr.openbsd.org
> user and password
> ascii art
> ftp> cd pub/Open?    <- Here press the tab key
https://en.wikipedia.org/wiki/%C2%BA
> segmentation fault (core dumped)  ftp ftp.fr.openbsd.org
>
> It also crashes with the letter "?" and "?".
https://en.wikipedia.org/wiki/%C3%81
https://en.wikipedia.org/wiki/%C3%91

>
> Tested in:
> - A snapshot from yesterday. i386. root account. console/ksh without
>   locales.
> - A snapshot from a few days ago. amd64. user. urxvt/zsh with utf8
>   locales.
>
> I also tested the bug in a remote session with OpenBSD 4.7 and ftp works
> without problems.
>
> I've updated the code of usr.bin/ftp to 2012-10-01 and 2012-01-01 and
> tried both versions. ftp also crashes.
>
> Backtrace:
> Thread 1 (process 3436):
> #0  memcpy (dst0=0x9d4160, src0=Variable "src0" is not available.
> ) at /usr/src/lib/libc/string/bcopy.c:115
> #1  0x000000000040432b in complete (el=Variable "el" is not available.
> ) at /usr/src/usr.bin/ftp/complete.c:313
> #2  0x000000000041eb84 in el_wgets (el=0x20da64800, nread=0x7f7ffffe3ebc) at read.c:612
> #3  0x000000000041ef8d in el_gets (el=0x20da64800, nread=Variable "nread" is not available.
> ) at eln.c:78
> #4  0x000000000040e55f in cmdscanner (top=Variable "top" is not available.
> ) at /usr/src/usr.bin/ftp/main.c:465
> #5  0x000000000040eb7c in main (argc=1, argv=0x7f7ffffe4398) at /usr/src/usr.bin/ftp/main.c:369
>
> Let me know if it's necessary more info or whatever :)
>
> Cheers.
>

--
Juan Francisco Cantero Hurtado http://juanfra.info

Reply | Threaded
Open this post in threaded view
|

Re: segfault in ftp

Otto Moerbeek
On Mon, Oct 29, 2012 at 06:43:13PM +0100, Juan Francisco Cantero Hurtado wrote:

> Chris Cappuccio sent me a mail saying he can't see the characters, only
> a question mark.
>
> I'm linking each character to their wikipedia page, so you can
> copy-paste the character.
>
> On Thu, Oct 25, 2012 at 05:07:34AM +0200, Juan Francisco Cantero Hurtado wrote:
> > This afternoon I was downloading a tarball from a OpenBSD mirror. I
> > press the key "?" and after the tab key. ftp crashed with a segfault.

Please also include your environment settings. It is likely locale
plays a role here.

At least env | grep LC

        -Otto

> https://en.wikipedia.org/wiki/%C2%BA
> >
> > Steps for reproduce:
> > # ftp ftp.fr.openbsd.org
> > user and password
> > ascii art
> > ftp> cd pub/Open?    <- Here press the tab key
> https://en.wikipedia.org/wiki/%C2%BA
> > segmentation fault (core dumped)  ftp ftp.fr.openbsd.org
> >
> > It also crashes with the letter "?" and "?".
> https://en.wikipedia.org/wiki/%C3%81
> https://en.wikipedia.org/wiki/%C3%91
> >
> > Tested in:
> > - A snapshot from yesterday. i386. root account. console/ksh without
> >   locales.
> > - A snapshot from a few days ago. amd64. user. urxvt/zsh with utf8
> >   locales.
> >
> > I also tested the bug in a remote session with OpenBSD 4.7 and ftp works
> > without problems.
> >
> > I've updated the code of usr.bin/ftp to 2012-10-01 and 2012-01-01 and
> > tried both versions. ftp also crashes.
> >
> > Backtrace:
> > Thread 1 (process 3436):
> > #0  memcpy (dst0=0x9d4160, src0=Variable "src0" is not available.
> > ) at /usr/src/lib/libc/string/bcopy.c:115
> > #1  0x000000000040432b in complete (el=Variable "el" is not available.
> > ) at /usr/src/usr.bin/ftp/complete.c:313
> > #2  0x000000000041eb84 in el_wgets (el=0x20da64800, nread=0x7f7ffffe3ebc) at read.c:612
> > #3  0x000000000041ef8d in el_gets (el=0x20da64800, nread=Variable "nread" is not available.
> > ) at eln.c:78
> > #4  0x000000000040e55f in cmdscanner (top=Variable "top" is not available.
> > ) at /usr/src/usr.bin/ftp/main.c:465
> > #5  0x000000000040eb7c in main (argc=1, argv=0x7f7ffffe4398) at /usr/src/usr.bin/ftp/main.c:369
> >
> > Let me know if it's necessary more info or whatever :)
> >
> > Cheers.
> >
>
> --
> Juan Francisco Cantero Hurtado http://juanfra.info

Reply | Threaded
Open this post in threaded view
|

Re: segfault in ftp

Juan Francisco Cantero Hurtado
On Tue, Oct 30, 2012 at 09:31:58AM +0100, Otto Moerbeek wrote:

> On Mon, Oct 29, 2012 at 06:43:13PM +0100, Juan Francisco Cantero Hurtado wrote:
>
> > Chris Cappuccio sent me a mail saying he can't see the characters, only
> > a question mark.
> >
> > I'm linking each character to their wikipedia page, so you can
> > copy-paste the character.
> >
> > On Thu, Oct 25, 2012 at 05:07:34AM +0200, Juan Francisco Cantero Hurtado wrote:
> > > This afternoon I was downloading a tarball from a OpenBSD mirror. I
> > > press the key "?" and after the tab key. ftp crashed with a segfault.
>
> Please also include your environment settings. It is likely locale
> plays a role here.
>
> At least env | grep LC
>

I've tried the bug in amd64 without locales and also with
LC_TIME="es_ES.ISO8859-1" LC_CTYPE="en_US.UTF-8".

The i386 system was a clean installation in a virtual machine.

>
> > https://en.wikipedia.org/wiki/%C2%BA
> > >
> > > Steps for reproduce:
> > > # ftp ftp.fr.openbsd.org
> > > user and password
> > > ascii art
> > > ftp> cd pub/Open?    <- Here press the tab key
> > https://en.wikipedia.org/wiki/%C2%BA
> > > segmentation fault (core dumped)  ftp ftp.fr.openbsd.org
> > >
> > > It also crashes with the letter "?" and "?".
> > https://en.wikipedia.org/wiki/%C3%81
> > https://en.wikipedia.org/wiki/%C3%91
> > >
> > > Tested in:
> > > - A snapshot from yesterday. i386. root account. console/ksh without
> > >   locales.
> > > - A snapshot from a few days ago. amd64. user. urxvt/zsh with utf8
> > >   locales.
> > >
> > > I also tested the bug in a remote session with OpenBSD 4.7 and ftp works
> > > without problems.
> > >
> > > I've updated the code of usr.bin/ftp to 2012-10-01 and 2012-01-01 and
> > > tried both versions. ftp also crashes.
> > >
> > > Backtrace:
> > > Thread 1 (process 3436):
> > > #0  memcpy (dst0=0x9d4160, src0=Variable "src0" is not available.
> > > ) at /usr/src/lib/libc/string/bcopy.c:115
> > > #1  0x000000000040432b in complete (el=Variable "el" is not available.
> > > ) at /usr/src/usr.bin/ftp/complete.c:313
> > > #2  0x000000000041eb84 in el_wgets (el=0x20da64800, nread=0x7f7ffffe3ebc) at read.c:612
> > > #3  0x000000000041ef8d in el_gets (el=0x20da64800, nread=Variable "nread" is not available.
> > > ) at eln.c:78
> > > #4  0x000000000040e55f in cmdscanner (top=Variable "top" is not available.
> > > ) at /usr/src/usr.bin/ftp/main.c:465
> > > #5  0x000000000040eb7c in main (argc=1, argv=0x7f7ffffe4398) at /usr/src/usr.bin/ftp/main.c:369
> > >
> > > Let me know if it's necessary more info or whatever :)
> > >
> > > Cheers.
> > >
> >

--
Juan Francisco Cantero Hurtado http://juanfra.info

Reply | Threaded
Open this post in threaded view
|

Re: segfault in ftp

Otto Moerbeek
On Tue, Oct 30, 2012 at 08:59:27PM +0100, Juan Francisco Cantero Hurtado wrote:

> On Tue, Oct 30, 2012 at 09:31:58AM +0100, Otto Moerbeek wrote:
> > On Mon, Oct 29, 2012 at 06:43:13PM +0100, Juan Francisco Cantero Hurtado wrote:
> >
> > > Chris Cappuccio sent me a mail saying he can't see the characters, only
> > > a question mark.
> > >
> > > I'm linking each character to their wikipedia page, so you can
> > > copy-paste the character.
> > >
> > > On Thu, Oct 25, 2012 at 05:07:34AM +0200, Juan Francisco Cantero Hurtado wrote:
> > > > This afternoon I was downloading a tarball from a OpenBSD mirror. I
> > > > press the key "?" and after the tab key. ftp crashed with a segfault.
> >
> > Please also include your environment settings. It is likely locale
> > plays a role here.
> >
> > At least env | grep LC
> >
>
> I've tried the bug in amd64 without locales and also with
> LC_TIME="es_ES.ISO8859-1" LC_CTYPE="en_US.UTF-8".
>
> The i386 system was a clean installation in a virtual machine.

I can now reproduce using a terminal that accepts more than just low ascii.

What I see is that when complete() is called the cursor position in
the EditLine struct is not what it is supposed to be, it points a
couple of bytes beyond the terminating NUL while it is supposed to
point to the NUL. That causes confusing in the scanner, getting the
argument list count wrong.

The root of the problem seems to be inside the editline lib.

Cc:ing nicm@, maybe he has a clue

        -Otto
       

>
> >
> > > https://en.wikipedia.org/wiki/%C2%BA
> > > >
> > > > Steps for reproduce:
> > > > # ftp ftp.fr.openbsd.org
> > > > user and password
> > > > ascii art
> > > > ftp> cd pub/Open?    <- Here press the tab key
> > > https://en.wikipedia.org/wiki/%C2%BA
> > > > segmentation fault (core dumped)  ftp ftp.fr.openbsd.org
> > > >
> > > > It also crashes with the letter "?" and "?".
> > > https://en.wikipedia.org/wiki/%C3%81
> > > https://en.wikipedia.org/wiki/%C3%91
> > > >
> > > > Tested in:
> > > > - A snapshot from yesterday. i386. root account. console/ksh without
> > > >   locales.
> > > > - A snapshot from a few days ago. amd64. user. urxvt/zsh with utf8
> > > >   locales.
> > > >
> > > > I also tested the bug in a remote session with OpenBSD 4.7 and ftp works
> > > > without problems.
> > > >
> > > > I've updated the code of usr.bin/ftp to 2012-10-01 and 2012-01-01 and
> > > > tried both versions. ftp also crashes.
> > > >
> > > > Backtrace:
> > > > Thread 1 (process 3436):
> > > > #0  memcpy (dst0=0x9d4160, src0=Variable "src0" is not available.
> > > > ) at /usr/src/lib/libc/string/bcopy.c:115
> > > > #1  0x000000000040432b in complete (el=Variable "el" is not available.
> > > > ) at /usr/src/usr.bin/ftp/complete.c:313
> > > > #2  0x000000000041eb84 in el_wgets (el=0x20da64800, nread=0x7f7ffffe3ebc) at read.c:612
> > > > #3  0x000000000041ef8d in el_gets (el=0x20da64800, nread=Variable "nread" is not available.
> > > > ) at eln.c:78
> > > > #4  0x000000000040e55f in cmdscanner (top=Variable "top" is not available.
> > > > ) at /usr/src/usr.bin/ftp/main.c:465
> > > > #5  0x000000000040eb7c in main (argc=1, argv=0x7f7ffffe4398) at /usr/src/usr.bin/ftp/main.c:369
> > > >
> > > > Let me know if it's necessary more info or whatever :)
> > > >
> > > > Cheers.
> > > >
> > >
>
> --
> Juan Francisco Cantero Hurtado http://juanfra.info

Reply | Threaded
Open this post in threaded view
|

Re: segfault in ftp

Otto Moerbeek
On Tue, Oct 30, 2012 at 10:17:12PM +0100, Otto Moerbeek wrote:

> On Tue, Oct 30, 2012 at 08:59:27PM +0100, Juan Francisco Cantero Hurtado wrote:
>
> > On Tue, Oct 30, 2012 at 09:31:58AM +0100, Otto Moerbeek wrote:
> > > On Mon, Oct 29, 2012 at 06:43:13PM +0100, Juan Francisco Cantero Hurtado wrote:
> > >
> > > > Chris Cappuccio sent me a mail saying he can't see the characters, only
> > > > a question mark.
> > > >
> > > > I'm linking each character to their wikipedia page, so you can
> > > > copy-paste the character.
> > > >
> > > > On Thu, Oct 25, 2012 at 05:07:34AM +0200, Juan Francisco Cantero Hurtado wrote:
> > > > > This afternoon I was downloading a tarball from a OpenBSD mirror. I
> > > > > press the key "?" and after the tab key. ftp crashed with a segfault.
> > >
> > > Please also include your environment settings. It is likely locale
> > > plays a role here.
> > >
> > > At least env | grep LC
> > >
> >
> > I've tried the bug in amd64 without locales and also with
> > LC_TIME="es_ES.ISO8859-1" LC_CTYPE="en_US.UTF-8".
> >
> > The i386 system was a clean installation in a virtual machine.
>
> I can now reproduce using a terminal that accepts more than just low ascii.
>
> What I see is that when complete() is called the cursor position in
> the EditLine struct is not what it is supposed to be, it points a
> couple of bytes beyond the terminating NUL while it is supposed to
> point to the NUL. That causes confusing in the scanner, getting the
> argument list count wrong.

Ehh, the buffer is not NUL terminated, but observation still holds:
the cursor position is a couple of bytes further than it
should be.

>
> The root of the problem seems to be inside the editline lib.
>
> Cc:ing nicm@, maybe he has a clue
>
> -Otto
>
>
> >
> > >
> > > > https://en.wikipedia.org/wiki/%C2%BA
> > > > >
> > > > > Steps for reproduce:
> > > > > # ftp ftp.fr.openbsd.org
> > > > > user and password
> > > > > ascii art
> > > > > ftp> cd pub/Open?    <- Here press the tab key
> > > > https://en.wikipedia.org/wiki/%C2%BA
> > > > > segmentation fault (core dumped)  ftp ftp.fr.openbsd.org
> > > > >
> > > > > It also crashes with the letter "?" and "?".
> > > > https://en.wikipedia.org/wiki/%C3%81
> > > > https://en.wikipedia.org/wiki/%C3%91
> > > > >
> > > > > Tested in:
> > > > > - A snapshot from yesterday. i386. root account. console/ksh without
> > > > >   locales.
> > > > > - A snapshot from a few days ago. amd64. user. urxvt/zsh with utf8
> > > > >   locales.
> > > > >
> > > > > I also tested the bug in a remote session with OpenBSD 4.7 and ftp works
> > > > > without problems.
> > > > >
> > > > > I've updated the code of usr.bin/ftp to 2012-10-01 and 2012-01-01 and
> > > > > tried both versions. ftp also crashes.
> > > > >
> > > > > Backtrace:
> > > > > Thread 1 (process 3436):
> > > > > #0  memcpy (dst0=0x9d4160, src0=Variable "src0" is not available.
> > > > > ) at /usr/src/lib/libc/string/bcopy.c:115
> > > > > #1  0x000000000040432b in complete (el=Variable "el" is not available.
> > > > > ) at /usr/src/usr.bin/ftp/complete.c:313
> > > > > #2  0x000000000041eb84 in el_wgets (el=0x20da64800, nread=0x7f7ffffe3ebc) at read.c:612
> > > > > #3  0x000000000041ef8d in el_gets (el=0x20da64800, nread=Variable "nread" is not available.
> > > > > ) at eln.c:78
> > > > > #4  0x000000000040e55f in cmdscanner (top=Variable "top" is not available.
> > > > > ) at /usr/src/usr.bin/ftp/main.c:465
> > > > > #5  0x000000000040eb7c in main (argc=1, argv=0x7f7ffffe4398) at /usr/src/usr.bin/ftp/main.c:369
> > > > >
> > > > > Let me know if it's necessary more info or whatever :)
> > > > >
> > > > > Cheers.
> > > > >
> > > >
> >
> > --
> > Juan Francisco Cantero Hurtado http://juanfra.info

Reply | Threaded
Open this post in threaded view
|

Re: segfault in ftp

Nicholas Marriott-2
Hi

The buffer isn't zero-terminated, it's the result of calling wctomb to
convert the internal wchar_t* that libedit has into a char*.

libedit works out the offset in el_line with ct_enc_width which rather
foolishly makes the assumption that wctomb will convert to UTF-8, but
ftp doesn't call setlocale so it just leaves it as ASCII.

Try this:

Index: main.c
===================================================================
RCS file: /cvs/src/usr.bin/ftp/main.c,v
retrieving revision 1.85
diff -u -p -r1.85 main.c
--- main.c 26 Aug 2012 02:16:02 -0000 1.85
+++ main.c 30 Oct 2012 23:52:34 -0000
@@ -67,6 +67,7 @@
 
 #include <ctype.h>
 #include <err.h>
+#include <locale.h>
 #include <netdb.h>
 #include <pwd.h>
 #include <stdio.h>
@@ -90,6 +91,8 @@ main(volatile int argc, char *argv[])
  char *outfile = NULL;
  const char *errstr;
  int dumb_terminal = 0;
+
+ setlocale(LC_CTYPE, "");
 
  ftpport = "ftp";
  httpport = "http";





On Tue, Oct 30, 2012 at 10:31:16PM +0100, Otto Moerbeek wrote:

> On Tue, Oct 30, 2012 at 10:17:12PM +0100, Otto Moerbeek wrote:
>
> > On Tue, Oct 30, 2012 at 08:59:27PM +0100, Juan Francisco Cantero Hurtado wrote:
> >
> > > On Tue, Oct 30, 2012 at 09:31:58AM +0100, Otto Moerbeek wrote:
> > > > On Mon, Oct 29, 2012 at 06:43:13PM +0100, Juan Francisco Cantero Hurtado wrote:
> > > >
> > > > > Chris Cappuccio sent me a mail saying he can't see the characters, only
> > > > > a question mark.
> > > > >
> > > > > I'm linking each character to their wikipedia page, so you can
> > > > > copy-paste the character.
> > > > >
> > > > > On Thu, Oct 25, 2012 at 05:07:34AM +0200, Juan Francisco Cantero Hurtado wrote:
> > > > > > This afternoon I was downloading a tarball from a OpenBSD mirror. I
> > > > > > press the key "?" and after the tab key. ftp crashed with a segfault.
> > > >
> > > > Please also include your environment settings. It is likely locale
> > > > plays a role here.
> > > >
> > > > At least env | grep LC
> > > >
> > >
> > > I've tried the bug in amd64 without locales and also with
> > > LC_TIME="es_ES.ISO8859-1" LC_CTYPE="en_US.UTF-8".
> > >
> > > The i386 system was a clean installation in a virtual machine.
> >
> > I can now reproduce using a terminal that accepts more than just low ascii.
> >
> > What I see is that when complete() is called the cursor position in
> > the EditLine struct is not what it is supposed to be, it points a
> > couple of bytes beyond the terminating NUL while it is supposed to
> > point to the NUL. That causes confusing in the scanner, getting the
> > argument list count wrong.
>
> Ehh, the buffer is not NUL terminated, but observation still holds:
> the cursor position is a couple of bytes further than it
> should be.
>
> >
> > The root of the problem seems to be inside the editline lib.
> >
> > Cc:ing nicm@, maybe he has a clue
> >
> > -Otto
> >
> >
> > >
> > > >
> > > > > https://en.wikipedia.org/wiki/%C2%BA
> > > > > >
> > > > > > Steps for reproduce:
> > > > > > # ftp ftp.fr.openbsd.org
> > > > > > user and password
> > > > > > ascii art
> > > > > > ftp> cd pub/Open?    <- Here press the tab key
> > > > > https://en.wikipedia.org/wiki/%C2%BA
> > > > > > segmentation fault (core dumped)  ftp ftp.fr.openbsd.org
> > > > > >
> > > > > > It also crashes with the letter "?" and "?".
> > > > > https://en.wikipedia.org/wiki/%C3%81
> > > > > https://en.wikipedia.org/wiki/%C3%91
> > > > > >
> > > > > > Tested in:
> > > > > > - A snapshot from yesterday. i386. root account. console/ksh without
> > > > > >   locales.
> > > > > > - A snapshot from a few days ago. amd64. user. urxvt/zsh with utf8
> > > > > >   locales.
> > > > > >
> > > > > > I also tested the bug in a remote session with OpenBSD 4.7 and ftp works
> > > > > > without problems.
> > > > > >
> > > > > > I've updated the code of usr.bin/ftp to 2012-10-01 and 2012-01-01 and
> > > > > > tried both versions. ftp also crashes.
> > > > > >
> > > > > > Backtrace:
> > > > > > Thread 1 (process 3436):
> > > > > > #0  memcpy (dst0=0x9d4160, src0=Variable "src0" is not available.
> > > > > > ) at /usr/src/lib/libc/string/bcopy.c:115
> > > > > > #1  0x000000000040432b in complete (el=Variable "el" is not available.
> > > > > > ) at /usr/src/usr.bin/ftp/complete.c:313
> > > > > > #2  0x000000000041eb84 in el_wgets (el=0x20da64800, nread=0x7f7ffffe3ebc) at read.c:612
> > > > > > #3  0x000000000041ef8d in el_gets (el=0x20da64800, nread=Variable "nread" is not available.
> > > > > > ) at eln.c:78
> > > > > > #4  0x000000000040e55f in cmdscanner (top=Variable "top" is not available.
> > > > > > ) at /usr/src/usr.bin/ftp/main.c:465
> > > > > > #5  0x000000000040eb7c in main (argc=1, argv=0x7f7ffffe4398) at /usr/src/usr.bin/ftp/main.c:369
> > > > > >
> > > > > > Let me know if it's necessary more info or whatever :)
> > > > > >
> > > > > > Cheers.
> > > > > >
> > > > >
> > >
> > > --
> > > Juan Francisco Cantero Hurtado http://juanfra.info

Reply | Threaded
Open this post in threaded view
|

Re: segfault in ftp

Nicholas Marriott-2
That is:

- libedit has a wchar_t* buffer (el->el_line.buffer) and el_line calls
  ct_encode_string to convert it to a char*.

- ct_encode_string calls wctomb which it expects to make UTF-8 but in
  fact because setlocale has not been called it outputs ASCII.

- el_line then uses ct_enc_width which assumes UTF-8 and returns 2. So
  the offset is adjusted by 2 even though only 1 byte was filled in.

- ftp obviously isn't happy about having a position after a \0, so it
  goes boom.

The setlocale() change below will only fix the problem if LC_CTYPE or
LC_ALL is set to UTF-8. ftp still cores if pasting UTF-8 in C locale.

I think the right fix is for libedit to use the return value of wctomb
to adjust the offset rather than assuming UTF-8 and working out the
width itself.

Perhaps something like this (very lightly tested):

Index: chartype.c
===================================================================
RCS file: /cvs/src/lib/libedit/chartype.c,v
retrieving revision 1.4
diff -u -p -r1.4 chartype.c
--- chartype.c 17 Nov 2011 20:14:24 -0000 1.4
+++ chartype.c 31 Oct 2012 00:13:12 -0000
@@ -44,6 +44,8 @@
 #define CT_BUFSIZ 1024
 
 #ifdef WIDECHAR
+protected ssize_t ct_encode_char1(char *, size_t, Char);
+
 protected void
 ct_conv_buff_resize(ct_buffer_t *conv, size_t mincsize, size_t minwsize)
 {
@@ -178,27 +180,25 @@ ct_decode_argv(int argc, const char *arg
 protected size_t
 ct_enc_width(Char c)
 {
- /* UTF-8 encoding specific values */
- if (c < 0x80)
- return 1;
- else if (c < 0x0800)
- return 2;
- else if (c < 0x10000)
- return 3;
- else if (c < 0x110000)
- return 4;
- else
- return 0; /* not a valid codepoint */
+ char s[MB_CUR_MAX];
+
+ return ct_encode_char1(s, sizeof s, c);
 }
 
 protected ssize_t
 ct_encode_char(char *dst, size_t len, Char c)
 {
- ssize_t l = 0;
  if (len < ct_enc_width(c))
  return -1;
- l = ct_wctomb(dst, c);
+ return ct_encode_char1(dst, len, c);
+}
 
+protected ssize_t
+ct_encode_char1(char *dst, size_t len, Char c)
+{
+ ssize_t l = 0;
+
+ l = ct_wctomb(dst, c);
  if (l < 0) {
  ct_wctomb_reset;
  l = 0;




On Tue, Oct 30, 2012 at 11:56:18PM +0000, Nicholas Marriott wrote:

> Hi
>
> The buffer isn't zero-terminated, it's the result of calling wctomb to
> convert the internal wchar_t* that libedit has into a char*.
>
> libedit works out the offset in el_line with ct_enc_width which rather
> foolishly makes the assumption that wctomb will convert to UTF-8, but
> ftp doesn't call setlocale so it just leaves it as ASCII.
>
> Try this:
>
> Index: main.c
> ===================================================================
> RCS file: /cvs/src/usr.bin/ftp/main.c,v
> retrieving revision 1.85
> diff -u -p -r1.85 main.c
> --- main.c 26 Aug 2012 02:16:02 -0000 1.85
> +++ main.c 30 Oct 2012 23:52:34 -0000
> @@ -67,6 +67,7 @@
>  
>  #include <ctype.h>
>  #include <err.h>
> +#include <locale.h>
>  #include <netdb.h>
>  #include <pwd.h>
>  #include <stdio.h>
> @@ -90,6 +91,8 @@ main(volatile int argc, char *argv[])
>   char *outfile = NULL;
>   const char *errstr;
>   int dumb_terminal = 0;
> +
> + setlocale(LC_CTYPE, "");
>  
>   ftpport = "ftp";
>   httpport = "http";
>
>
>
>
>
> On Tue, Oct 30, 2012 at 10:31:16PM +0100, Otto Moerbeek wrote:
> > On Tue, Oct 30, 2012 at 10:17:12PM +0100, Otto Moerbeek wrote:
> >
> > > On Tue, Oct 30, 2012 at 08:59:27PM +0100, Juan Francisco Cantero Hurtado wrote:
> > >
> > > > On Tue, Oct 30, 2012 at 09:31:58AM +0100, Otto Moerbeek wrote:
> > > > > On Mon, Oct 29, 2012 at 06:43:13PM +0100, Juan Francisco Cantero Hurtado wrote:
> > > > >
> > > > > > Chris Cappuccio sent me a mail saying he can't see the characters, only
> > > > > > a question mark.
> > > > > >
> > > > > > I'm linking each character to their wikipedia page, so you can
> > > > > > copy-paste the character.
> > > > > >
> > > > > > On Thu, Oct 25, 2012 at 05:07:34AM +0200, Juan Francisco Cantero Hurtado wrote:
> > > > > > > This afternoon I was downloading a tarball from a OpenBSD mirror. I
> > > > > > > press the key "?" and after the tab key. ftp crashed with a segfault.
> > > > >
> > > > > Please also include your environment settings. It is likely locale
> > > > > plays a role here.
> > > > >
> > > > > At least env | grep LC
> > > > >
> > > >
> > > > I've tried the bug in amd64 without locales and also with
> > > > LC_TIME="es_ES.ISO8859-1" LC_CTYPE="en_US.UTF-8".
> > > >
> > > > The i386 system was a clean installation in a virtual machine.
> > >
> > > I can now reproduce using a terminal that accepts more than just low ascii.
> > >
> > > What I see is that when complete() is called the cursor position in
> > > the EditLine struct is not what it is supposed to be, it points a
> > > couple of bytes beyond the terminating NUL while it is supposed to
> > > point to the NUL. That causes confusing in the scanner, getting the
> > > argument list count wrong.
> >
> > Ehh, the buffer is not NUL terminated, but observation still holds:
> > the cursor position is a couple of bytes further than it
> > should be.
> >
> > >
> > > The root of the problem seems to be inside the editline lib.
> > >
> > > Cc:ing nicm@, maybe he has a clue
> > >
> > > -Otto
> > >
> > >
> > > >
> > > > >
> > > > > > https://en.wikipedia.org/wiki/%C2%BA
> > > > > > >
> > > > > > > Steps for reproduce:
> > > > > > > # ftp ftp.fr.openbsd.org
> > > > > > > user and password
> > > > > > > ascii art
> > > > > > > ftp> cd pub/Open?    <- Here press the tab key
> > > > > > https://en.wikipedia.org/wiki/%C2%BA
> > > > > > > segmentation fault (core dumped)  ftp ftp.fr.openbsd.org
> > > > > > >
> > > > > > > It also crashes with the letter "?" and "?".
> > > > > > https://en.wikipedia.org/wiki/%C3%81
> > > > > > https://en.wikipedia.org/wiki/%C3%91
> > > > > > >
> > > > > > > Tested in:
> > > > > > > - A snapshot from yesterday. i386. root account. console/ksh without
> > > > > > >   locales.
> > > > > > > - A snapshot from a few days ago. amd64. user. urxvt/zsh with utf8
> > > > > > >   locales.
> > > > > > >
> > > > > > > I also tested the bug in a remote session with OpenBSD 4.7 and ftp works
> > > > > > > without problems.
> > > > > > >
> > > > > > > I've updated the code of usr.bin/ftp to 2012-10-01 and 2012-01-01 and
> > > > > > > tried both versions. ftp also crashes.
> > > > > > >
> > > > > > > Backtrace:
> > > > > > > Thread 1 (process 3436):
> > > > > > > #0  memcpy (dst0=0x9d4160, src0=Variable "src0" is not available.
> > > > > > > ) at /usr/src/lib/libc/string/bcopy.c:115
> > > > > > > #1  0x000000000040432b in complete (el=Variable "el" is not available.
> > > > > > > ) at /usr/src/usr.bin/ftp/complete.c:313
> > > > > > > #2  0x000000000041eb84 in el_wgets (el=0x20da64800, nread=0x7f7ffffe3ebc) at read.c:612
> > > > > > > #3  0x000000000041ef8d in el_gets (el=0x20da64800, nread=Variable "nread" is not available.
> > > > > > > ) at eln.c:78
> > > > > > > #4  0x000000000040e55f in cmdscanner (top=Variable "top" is not available.
> > > > > > > ) at /usr/src/usr.bin/ftp/main.c:465
> > > > > > > #5  0x000000000040eb7c in main (argc=1, argv=0x7f7ffffe4398) at /usr/src/usr.bin/ftp/main.c:369
> > > > > > >
> > > > > > > Let me know if it's necessary more info or whatever :)
> > > > > > >
> > > > > > > Cheers.
> > > > > > >
> > > > > >
> > > >
> > > > --
> > > > Juan Francisco Cantero Hurtado http://juanfra.info

Reply | Threaded
Open this post in threaded view
|

Re: segfault in ftp

Juan Francisco Cantero Hurtado
In reply to this post by Nicholas Marriott-2
On Tue, Oct 30, 2012 at 11:56:18PM +0000, Nicholas Marriott wrote:

> Hi
>
> The buffer isn't zero-terminated, it's the result of calling wctomb to
> convert the internal wchar_t* that libedit has into a char*.
>
> libedit works out the offset in el_line with ct_enc_width which rather
> foolishly makes the assumption that wctomb will convert to UTF-8, but
> ftp doesn't call setlocale so it just leaves it as ASCII.
>
> Try this:
>
> Index: main.c
> ===================================================================
> RCS file: /cvs/src/usr.bin/ftp/main.c,v
> retrieving revision 1.85
> diff -u -p -r1.85 main.c
> --- main.c 26 Aug 2012 02:16:02 -0000 1.85
> +++ main.c 30 Oct 2012 23:52:34 -0000
> @@ -67,6 +67,7 @@
>  
>  #include <ctype.h>
>  #include <err.h>
> +#include <locale.h>
>  #include <netdb.h>
>  #include <pwd.h>
>  #include <stdio.h>
> @@ -90,6 +91,8 @@ main(volatile int argc, char *argv[])
>   char *outfile = NULL;
>   const char *errstr;
>   int dumb_terminal = 0;
> +
> + setlocale(LC_CTYPE, "");
>  
>   ftpport = "ftp";
>   httpport = "http";
>

Your patch fix the bug. Thanks a lot!

>
> On Tue, Oct 30, 2012 at 10:31:16PM +0100, Otto Moerbeek wrote:
> > On Tue, Oct 30, 2012 at 10:17:12PM +0100, Otto Moerbeek wrote:
> >
> > > On Tue, Oct 30, 2012 at 08:59:27PM +0100, Juan Francisco Cantero Hurtado wrote:
> > >
> > > > On Tue, Oct 30, 2012 at 09:31:58AM +0100, Otto Moerbeek wrote:
> > > > > On Mon, Oct 29, 2012 at 06:43:13PM +0100, Juan Francisco Cantero Hurtado wrote:
> > > > >
> > > > > > Chris Cappuccio sent me a mail saying he can't see the characters, only
> > > > > > a question mark.
> > > > > >
> > > > > > I'm linking each character to their wikipedia page, so you can
> > > > > > copy-paste the character.
> > > > > >
> > > > > > On Thu, Oct 25, 2012 at 05:07:34AM +0200, Juan Francisco Cantero Hurtado wrote:
> > > > > > > This afternoon I was downloading a tarball from a OpenBSD mirror. I
> > > > > > > press the key "?" and after the tab key. ftp crashed with a segfault.
> > > > >
> > > > > Please also include your environment settings. It is likely locale
> > > > > plays a role here.
> > > > >
> > > > > At least env | grep LC
> > > > >
> > > >
> > > > I've tried the bug in amd64 without locales and also with
> > > > LC_TIME="es_ES.ISO8859-1" LC_CTYPE="en_US.UTF-8".
> > > >
> > > > The i386 system was a clean installation in a virtual machine.
> > >
> > > I can now reproduce using a terminal that accepts more than just low ascii.
> > >
> > > What I see is that when complete() is called the cursor position in
> > > the EditLine struct is not what it is supposed to be, it points a
> > > couple of bytes beyond the terminating NUL while it is supposed to
> > > point to the NUL. That causes confusing in the scanner, getting the
> > > argument list count wrong.
> >
> > Ehh, the buffer is not NUL terminated, but observation still holds:
> > the cursor position is a couple of bytes further than it
> > should be.
> >
> > >
> > > The root of the problem seems to be inside the editline lib.
> > >
> > > Cc:ing nicm@, maybe he has a clue
> > >
> > > -Otto
> > >
> > >
> > > >
> > > > >
> > > > > > https://en.wikipedia.org/wiki/%C2%BA
> > > > > > >
> > > > > > > Steps for reproduce:
> > > > > > > # ftp ftp.fr.openbsd.org
> > > > > > > user and password
> > > > > > > ascii art
> > > > > > > ftp> cd pub/Open?    <- Here press the tab key
> > > > > > https://en.wikipedia.org/wiki/%C2%BA
> > > > > > > segmentation fault (core dumped)  ftp ftp.fr.openbsd.org
> > > > > > >
> > > > > > > It also crashes with the letter "?" and "?".
> > > > > > https://en.wikipedia.org/wiki/%C3%81
> > > > > > https://en.wikipedia.org/wiki/%C3%91
> > > > > > >
> > > > > > > Tested in:
> > > > > > > - A snapshot from yesterday. i386. root account. console/ksh without
> > > > > > >   locales.
> > > > > > > - A snapshot from a few days ago. amd64. user. urxvt/zsh with utf8
> > > > > > >   locales.
> > > > > > >
> > > > > > > I also tested the bug in a remote session with OpenBSD 4.7 and ftp works
> > > > > > > without problems.
> > > > > > >
> > > > > > > I've updated the code of usr.bin/ftp to 2012-10-01 and 2012-01-01 and
> > > > > > > tried both versions. ftp also crashes.
> > > > > > >
> > > > > > > Backtrace:
> > > > > > > Thread 1 (process 3436):
> > > > > > > #0  memcpy (dst0=0x9d4160, src0=Variable "src0" is not available.
> > > > > > > ) at /usr/src/lib/libc/string/bcopy.c:115
> > > > > > > #1  0x000000000040432b in complete (el=Variable "el" is not available.
> > > > > > > ) at /usr/src/usr.bin/ftp/complete.c:313
> > > > > > > #2  0x000000000041eb84 in el_wgets (el=0x20da64800, nread=0x7f7ffffe3ebc) at read.c:612
> > > > > > > #3  0x000000000041ef8d in el_gets (el=0x20da64800, nread=Variable "nread" is not available.
> > > > > > > ) at eln.c:78
> > > > > > > #4  0x000000000040e55f in cmdscanner (top=Variable "top" is not available.
> > > > > > > ) at /usr/src/usr.bin/ftp/main.c:465
> > > > > > > #5  0x000000000040eb7c in main (argc=1, argv=0x7f7ffffe4398) at /usr/src/usr.bin/ftp/main.c:369
> > > > > > >
> > > > > > > Let me know if it's necessary more info or whatever :)
> > > > > > >
> > > > > > > Cheers.
> > > > > > >
> > > > > >

--
Juan Francisco Cantero Hurtado http://juanfra.info

Reply | Threaded
Open this post in threaded view
|

Re: segfault in ftp

Juan Francisco Cantero Hurtado
In reply to this post by Nicholas Marriott-2
On Wed, Oct 31, 2012 at 12:13:56AM +0000, Nicholas Marriott wrote:

> That is:
>
> - libedit has a wchar_t* buffer (el->el_line.buffer) and el_line calls
>   ct_encode_string to convert it to a char*.
>
> - ct_encode_string calls wctomb which it expects to make UTF-8 but in
>   fact because setlocale has not been called it outputs ASCII.
>
> - el_line then uses ct_enc_width which assumes UTF-8 and returns 2. So
>   the offset is adjusted by 2 even though only 1 byte was filled in.
>
> - ftp obviously isn't happy about having a position after a \0, so it
>   goes boom.
>
> The setlocale() change below will only fix the problem if LC_CTYPE or
> LC_ALL is set to UTF-8. ftp still cores if pasting UTF-8 in C locale.
>
> I think the right fix is for libedit to use the return value of wctomb
> to adjust the offset rather than assuming UTF-8 and working out the
> width itself.
>
> Perhaps something like this (very lightly tested):
>
> Index: chartype.c
> ===================================================================
> RCS file: /cvs/src/lib/libedit/chartype.c,v
> retrieving revision 1.4
> diff -u -p -r1.4 chartype.c
> --- chartype.c 17 Nov 2011 20:14:24 -0000 1.4
> +++ chartype.c 31 Oct 2012 00:13:12 -0000
> @@ -44,6 +44,8 @@
>  #define CT_BUFSIZ 1024
>  
>  #ifdef WIDECHAR
> +protected ssize_t ct_encode_char1(char *, size_t, Char);
> +
>  protected void
>  ct_conv_buff_resize(ct_buffer_t *conv, size_t mincsize, size_t minwsize)
>  {
> @@ -178,27 +180,25 @@ ct_decode_argv(int argc, const char *arg
>  protected size_t
>  ct_enc_width(Char c)
>  {
> - /* UTF-8 encoding specific values */
> - if (c < 0x80)
> - return 1;
> - else if (c < 0x0800)
> - return 2;
> - else if (c < 0x10000)
> - return 3;
> - else if (c < 0x110000)
> - return 4;
> - else
> - return 0; /* not a valid codepoint */
> + char s[MB_CUR_MAX];
> +
> + return ct_encode_char1(s, sizeof s, c);
>  }
>  
>  protected ssize_t
>  ct_encode_char(char *dst, size_t len, Char c)
>  {
> - ssize_t l = 0;
>   if (len < ct_enc_width(c))
>   return -1;
> - l = ct_wctomb(dst, c);
> + return ct_encode_char1(dst, len, c);
> +}
>  
> +protected ssize_t
> +ct_encode_char1(char *dst, size_t len, Char c)
> +{
> + ssize_t l = 0;
> +
> + l = ct_wctomb(dst, c);
>   if (l < 0) {
>   ct_wctomb_reset;
>   l = 0;
>

With this patch (and without the patch for ftp), the behavior of ftp is
a little weird.

> cd pub/Open
             ^

Here I write https://en.wikipedia.org/wiki/%C2%BA and press tab. I can
see the character. I press backspace and the completion doesn't work. I
press backspace again and tab. ftp completes to "cd pub/OpeBSD".


>
> On Tue, Oct 30, 2012 at 11:56:18PM +0000, Nicholas Marriott wrote:
> > Hi
> >
> > The buffer isn't zero-terminated, it's the result of calling wctomb to
> > convert the internal wchar_t* that libedit has into a char*.
> >
> > libedit works out the offset in el_line with ct_enc_width which rather
> > foolishly makes the assumption that wctomb will convert to UTF-8, but
> > ftp doesn't call setlocale so it just leaves it as ASCII.
> >
> > Try this:
> >
> > Index: main.c
> > ===================================================================
> > RCS file: /cvs/src/usr.bin/ftp/main.c,v
> > retrieving revision 1.85
> > diff -u -p -r1.85 main.c
> > --- main.c 26 Aug 2012 02:16:02 -0000 1.85
> > +++ main.c 30 Oct 2012 23:52:34 -0000
> > @@ -67,6 +67,7 @@
> >  
> >  #include <ctype.h>
> >  #include <err.h>
> > +#include <locale.h>
> >  #include <netdb.h>
> >  #include <pwd.h>
> >  #include <stdio.h>
> > @@ -90,6 +91,8 @@ main(volatile int argc, char *argv[])
> >   char *outfile = NULL;
> >   const char *errstr;
> >   int dumb_terminal = 0;
> > +
> > + setlocale(LC_CTYPE, "");
> >  
> >   ftpport = "ftp";
> >   httpport = "http";
> >
> >
> >
> >
> >
> > On Tue, Oct 30, 2012 at 10:31:16PM +0100, Otto Moerbeek wrote:
> > > On Tue, Oct 30, 2012 at 10:17:12PM +0100, Otto Moerbeek wrote:
> > >
> > > > On Tue, Oct 30, 2012 at 08:59:27PM +0100, Juan Francisco Cantero Hurtado wrote:
> > > >
> > > > > On Tue, Oct 30, 2012 at 09:31:58AM +0100, Otto Moerbeek wrote:
> > > > > > On Mon, Oct 29, 2012 at 06:43:13PM +0100, Juan Francisco Cantero Hurtado wrote:
> > > > > >
> > > > > > > Chris Cappuccio sent me a mail saying he can't see the characters, only
> > > > > > > a question mark.
> > > > > > >
> > > > > > > I'm linking each character to their wikipedia page, so you can
> > > > > > > copy-paste the character.
> > > > > > >
> > > > > > > On Thu, Oct 25, 2012 at 05:07:34AM +0200, Juan Francisco Cantero Hurtado wrote:
> > > > > > > > This afternoon I was downloading a tarball from a OpenBSD mirror. I
> > > > > > > > press the key "?" and after the tab key. ftp crashed with a segfault.
> > > > > >
> > > > > > Please also include your environment settings. It is likely locale
> > > > > > plays a role here.
> > > > > >
> > > > > > At least env | grep LC
> > > > > >
> > > > >
> > > > > I've tried the bug in amd64 without locales and also with
> > > > > LC_TIME="es_ES.ISO8859-1" LC_CTYPE="en_US.UTF-8".
> > > > >
> > > > > The i386 system was a clean installation in a virtual machine.
> > > >
> > > > I can now reproduce using a terminal that accepts more than just low ascii.
> > > >
> > > > What I see is that when complete() is called the cursor position in
> > > > the EditLine struct is not what it is supposed to be, it points a
> > > > couple of bytes beyond the terminating NUL while it is supposed to
> > > > point to the NUL. That causes confusing in the scanner, getting the
> > > > argument list count wrong.
> > >
> > > Ehh, the buffer is not NUL terminated, but observation still holds:
> > > the cursor position is a couple of bytes further than it
> > > should be.
> > >
> > > >
> > > > The root of the problem seems to be inside the editline lib.
> > > >
> > > > Cc:ing nicm@, maybe he has a clue
> > > >
> > > > -Otto
> > > >
> > > >
> > > > >
> > > > > >
> > > > > > > https://en.wikipedia.org/wiki/%C2%BA
> > > > > > > >
> > > > > > > > Steps for reproduce:
> > > > > > > > # ftp ftp.fr.openbsd.org
> > > > > > > > user and password
> > > > > > > > ascii art
> > > > > > > > ftp> cd pub/Open?    <- Here press the tab key
> > > > > > > https://en.wikipedia.org/wiki/%C2%BA
> > > > > > > > segmentation fault (core dumped)  ftp ftp.fr.openbsd.org
> > > > > > > >
> > > > > > > > It also crashes with the letter "?" and "?".
> > > > > > > https://en.wikipedia.org/wiki/%C3%81
> > > > > > > https://en.wikipedia.org/wiki/%C3%91
> > > > > > > >
> > > > > > > > Tested in:
> > > > > > > > - A snapshot from yesterday. i386. root account. console/ksh without
> > > > > > > >   locales.
> > > > > > > > - A snapshot from a few days ago. amd64. user. urxvt/zsh with utf8
> > > > > > > >   locales.
> > > > > > > >
> > > > > > > > I also tested the bug in a remote session with OpenBSD 4.7 and ftp works
> > > > > > > > without problems.
> > > > > > > >
> > > > > > > > I've updated the code of usr.bin/ftp to 2012-10-01 and 2012-01-01 and
> > > > > > > > tried both versions. ftp also crashes.
> > > > > > > >
> > > > > > > > Backtrace:
> > > > > > > > Thread 1 (process 3436):
> > > > > > > > #0  memcpy (dst0=0x9d4160, src0=Variable "src0" is not available.
> > > > > > > > ) at /usr/src/lib/libc/string/bcopy.c:115
> > > > > > > > #1  0x000000000040432b in complete (el=Variable "el" is not available.
> > > > > > > > ) at /usr/src/usr.bin/ftp/complete.c:313
> > > > > > > > #2  0x000000000041eb84 in el_wgets (el=0x20da64800, nread=0x7f7ffffe3ebc) at read.c:612
> > > > > > > > #3  0x000000000041ef8d in el_gets (el=0x20da64800, nread=Variable "nread" is not available.
> > > > > > > > ) at eln.c:78
> > > > > > > > #4  0x000000000040e55f in cmdscanner (top=Variable "top" is not available.
> > > > > > > > ) at /usr/src/usr.bin/ftp/main.c:465
> > > > > > > > #5  0x000000000040eb7c in main (argc=1, argv=0x7f7ffffe4398) at /usr/src/usr.bin/ftp/main.c:369
> > > > > > > >
> > > > > > > > Let me know if it's necessary more info or whatever :)
> > > > > > > >
> > > > > > > > Cheers.
> > > > > > > >
> > > > > > >

--
Juan Francisco Cantero Hurtado http://juanfra.info

Reply | Threaded
Open this post in threaded view
|

Re: segfault in ftp

Otto Moerbeek
On Wed, Oct 31, 2012 at 02:14:34AM +0100, Juan Francisco Cantero Hurtado wrote:

> On Wed, Oct 31, 2012 at 12:13:56AM +0000, Nicholas Marriott wrote:
> > That is:
> >
> > - libedit has a wchar_t* buffer (el->el_line.buffer) and el_line calls
> >   ct_encode_string to convert it to a char*.
> >
> > - ct_encode_string calls wctomb which it expects to make UTF-8 but in
> >   fact because setlocale has not been called it outputs ASCII.
> >
> > - el_line then uses ct_enc_width which assumes UTF-8 and returns 2. So
> >   the offset is adjusted by 2 even though only 1 byte was filled in.
> >
> > - ftp obviously isn't happy about having a position after a \0, so it
> >   goes boom.
> >
> > The setlocale() change below will only fix the problem if LC_CTYPE or
> > LC_ALL is set to UTF-8. ftp still cores if pasting UTF-8 in C locale.
> >
> > I think the right fix is for libedit to use the return value of wctomb
> > to adjust the offset rather than assuming UTF-8 and working out the
> > width itself.
> >
> > Perhaps something like this (very lightly tested):
> >
> > Index: chartype.c
> > ===================================================================
> > RCS file: /cvs/src/lib/libedit/chartype.c,v
> > retrieving revision 1.4
> > diff -u -p -r1.4 chartype.c
> > --- chartype.c 17 Nov 2011 20:14:24 -0000 1.4
> > +++ chartype.c 31 Oct 2012 00:13:12 -0000
> > @@ -44,6 +44,8 @@
> >  #define CT_BUFSIZ 1024
> >  
> >  #ifdef WIDECHAR
> > +protected ssize_t ct_encode_char1(char *, size_t, Char);
> > +
> >  protected void
> >  ct_conv_buff_resize(ct_buffer_t *conv, size_t mincsize, size_t minwsize)
> >  {
> > @@ -178,27 +180,25 @@ ct_decode_argv(int argc, const char *arg
> >  protected size_t
> >  ct_enc_width(Char c)
> >  {
> > - /* UTF-8 encoding specific values */
> > - if (c < 0x80)
> > - return 1;
> > - else if (c < 0x0800)
> > - return 2;
> > - else if (c < 0x10000)
> > - return 3;
> > - else if (c < 0x110000)
> > - return 4;
> > - else
> > - return 0; /* not a valid codepoint */
> > + char s[MB_CUR_MAX];
> > +
> > + return ct_encode_char1(s, sizeof s, c);
> >  }
> >  
> >  protected ssize_t
> >  ct_encode_char(char *dst, size_t len, Char c)
> >  {
> > - ssize_t l = 0;
> >   if (len < ct_enc_width(c))
> >   return -1;
> > - l = ct_wctomb(dst, c);
> > + return ct_encode_char1(dst, len, c);
> > +}
> >  
> > +protected ssize_t
> > +ct_encode_char1(char *dst, size_t len, Char c)
> > +{
> > + ssize_t l = 0;
> > +
> > + l = ct_wctomb(dst, c);
> >   if (l < 0) {
> >   ct_wctomb_reset;
> >   l = 0;
> >
>
> With this patch (and without the patch for ftp), the behavior of ftp is
> a little weird.
>
> > cd pub/Open
>              ^
>
> Here I write https://en.wikipedia.org/wiki/%C2%BA and press tab. I can
> see the character. I press backspace and the completion doesn't work. I
> press backspace again and tab. ftp completes to "cd pub/OpeBSD".

I think this is more or less expected. ftp knows nothing about utf8 or
other wide char stuff. It would be a major pain to teach ftp that (the
code is ugly ugly ugly). I think the best we can aim for now is not
crashing.

As for libedit, yestderday i was greatly confusd by the implemenattion
el_line(). But I'm happy it's actually libedit itself that is confused
about encodings. Will try to look at the diff later.

        -Otto

>
>
> >
> > On Tue, Oct 30, 2012 at 11:56:18PM +0000, Nicholas Marriott wrote:
> > > Hi
> > >
> > > The buffer isn't zero-terminated, it's the result of calling wctomb to
> > > convert the internal wchar_t* that libedit has into a char*.
> > >
> > > libedit works out the offset in el_line with ct_enc_width which rather
> > > foolishly makes the assumption that wctomb will convert to UTF-8, but
> > > ftp doesn't call setlocale so it just leaves it as ASCII.
> > >
> > > Try this:
> > >
> > > Index: main.c
> > > ===================================================================
> > > RCS file: /cvs/src/usr.bin/ftp/main.c,v
> > > retrieving revision 1.85
> > > diff -u -p -r1.85 main.c
> > > --- main.c 26 Aug 2012 02:16:02 -0000 1.85
> > > +++ main.c 30 Oct 2012 23:52:34 -0000
> > > @@ -67,6 +67,7 @@
> > >  
> > >  #include <ctype.h>
> > >  #include <err.h>
> > > +#include <locale.h>
> > >  #include <netdb.h>
> > >  #include <pwd.h>
> > >  #include <stdio.h>
> > > @@ -90,6 +91,8 @@ main(volatile int argc, char *argv[])
> > >   char *outfile = NULL;
> > >   const char *errstr;
> > >   int dumb_terminal = 0;
> > > +
> > > + setlocale(LC_CTYPE, "");
> > >  
> > >   ftpport = "ftp";
> > >   httpport = "http";
> > >
> > >
> > >
> > >
> > >
> > > On Tue, Oct 30, 2012 at 10:31:16PM +0100, Otto Moerbeek wrote:
> > > > On Tue, Oct 30, 2012 at 10:17:12PM +0100, Otto Moerbeek wrote:
> > > >
> > > > > On Tue, Oct 30, 2012 at 08:59:27PM +0100, Juan Francisco Cantero Hurtado wrote:
> > > > >
> > > > > > On Tue, Oct 30, 2012 at 09:31:58AM +0100, Otto Moerbeek wrote:
> > > > > > > On Mon, Oct 29, 2012 at 06:43:13PM +0100, Juan Francisco Cantero Hurtado wrote:
> > > > > > >
> > > > > > > > Chris Cappuccio sent me a mail saying he can't see the characters, only
> > > > > > > > a question mark.
> > > > > > > >
> > > > > > > > I'm linking each character to their wikipedia page, so you can
> > > > > > > > copy-paste the character.
> > > > > > > >
> > > > > > > > On Thu, Oct 25, 2012 at 05:07:34AM +0200, Juan Francisco Cantero Hurtado wrote:
> > > > > > > > > This afternoon I was downloading a tarball from a OpenBSD mirror. I
> > > > > > > > > press the key "?" and after the tab key. ftp crashed with a segfault.
> > > > > > >
> > > > > > > Please also include your environment settings. It is likely locale
> > > > > > > plays a role here.
> > > > > > >
> > > > > > > At least env | grep LC
> > > > > > >
> > > > > >
> > > > > > I've tried the bug in amd64 without locales and also with
> > > > > > LC_TIME="es_ES.ISO8859-1" LC_CTYPE="en_US.UTF-8".
> > > > > >
> > > > > > The i386 system was a clean installation in a virtual machine.
> > > > >
> > > > > I can now reproduce using a terminal that accepts more than just low ascii.
> > > > >
> > > > > What I see is that when complete() is called the cursor position in
> > > > > the EditLine struct is not what it is supposed to be, it points a
> > > > > couple of bytes beyond the terminating NUL while it is supposed to
> > > > > point to the NUL. That causes confusing in the scanner, getting the
> > > > > argument list count wrong.
> > > >
> > > > Ehh, the buffer is not NUL terminated, but observation still holds:
> > > > the cursor position is a couple of bytes further than it
> > > > should be.
> > > >
> > > > >
> > > > > The root of the problem seems to be inside the editline lib.
> > > > >
> > > > > Cc:ing nicm@, maybe he has a clue
> > > > >
> > > > > -Otto
> > > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > https://en.wikipedia.org/wiki/%C2%BA
> > > > > > > > >
> > > > > > > > > Steps for reproduce:
> > > > > > > > > # ftp ftp.fr.openbsd.org
> > > > > > > > > user and password
> > > > > > > > > ascii art
> > > > > > > > > ftp> cd pub/Open?    <- Here press the tab key
> > > > > > > > https://en.wikipedia.org/wiki/%C2%BA
> > > > > > > > > segmentation fault (core dumped)  ftp ftp.fr.openbsd.org
> > > > > > > > >
> > > > > > > > > It also crashes with the letter "?" and "?".
> > > > > > > > https://en.wikipedia.org/wiki/%C3%81
> > > > > > > > https://en.wikipedia.org/wiki/%C3%91
> > > > > > > > >
> > > > > > > > > Tested in:
> > > > > > > > > - A snapshot from yesterday. i386. root account. console/ksh without
> > > > > > > > >   locales.
> > > > > > > > > - A snapshot from a few days ago. amd64. user. urxvt/zsh with utf8
> > > > > > > > >   locales.
> > > > > > > > >
> > > > > > > > > I also tested the bug in a remote session with OpenBSD 4.7 and ftp works
> > > > > > > > > without problems.
> > > > > > > > >
> > > > > > > > > I've updated the code of usr.bin/ftp to 2012-10-01 and 2012-01-01 and
> > > > > > > > > tried both versions. ftp also crashes.
> > > > > > > > >
> > > > > > > > > Backtrace:
> > > > > > > > > Thread 1 (process 3436):
> > > > > > > > > #0  memcpy (dst0=0x9d4160, src0=Variable "src0" is not available.
> > > > > > > > > ) at /usr/src/lib/libc/string/bcopy.c:115
> > > > > > > > > #1  0x000000000040432b in complete (el=Variable "el" is not available.
> > > > > > > > > ) at /usr/src/usr.bin/ftp/complete.c:313
> > > > > > > > > #2  0x000000000041eb84 in el_wgets (el=0x20da64800, nread=0x7f7ffffe3ebc) at read.c:612
> > > > > > > > > #3  0x000000000041ef8d in el_gets (el=0x20da64800, nread=Variable "nread" is not available.
> > > > > > > > > ) at eln.c:78
> > > > > > > > > #4  0x000000000040e55f in cmdscanner (top=Variable "top" is not available.
> > > > > > > > > ) at /usr/src/usr.bin/ftp/main.c:465
> > > > > > > > > #5  0x000000000040eb7c in main (argc=1, argv=0x7f7ffffe4398) at /usr/src/usr.bin/ftp/main.c:369
> > > > > > > > >
> > > > > > > > > Let me know if it's necessary more info or whatever :)
> > > > > > > > >
> > > > > > > > > Cheers.
> > > > > > > > >
> > > > > > > >
>
> --
> Juan Francisco Cantero Hurtado http://juanfra.info