xterm not showing Norwegian alphabet

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

xterm not showing Norwegian alphabet

Xianwen Chen-3
Dear list,
I have a freshly installed OpenBSD 6.2 i386, with the following
environmental setting:

$env

LOGNAME=oe
WINDOWPATH=ttyC4
HOME=/home/oe
DISPLAY=:0
LC_ALL=en_US.UTF-8
PATH=/home/xc/bin:/bin:/sbin:/usr/bin:/usr/sbin:/usr/X11R6/bin:/usr/local/bin:/usr/local/sbin:/usr/games:.
SHELL=/bin/csh
USER=oe
TERM=xterm
WINDOWID=8388621
XTERM_VERSION=XTerm/OpenBSD(330)
XTERM_LOCALE=en_US.UTF-8
XTERM_SHELL=/bin/csh
PWD=/home/oe
LC_CTYPE=en_US.UTF-8
EXINIT=set ai sm noeb

I am however not able to show or type Norwegian alphabet of ø, å, and
æ. ø is shown as C8. å is shown as C%. And æ is shown is C&.

I tried to copi ø, å, and æ from SeaMonkey to xterm. The same thing happens.

I tried to copy C8, C%, and C& from xterm back to SeaMonkey. They
remained two characters as C8, C%, and C&.

Please help.

Sincerely,
Xianwen

Reply | Threaded
Open this post in threaded view
|

Re: xterm not showing Norwegian alphabet

Erling Westenvik-2
On Sat, Feb 10, 2018 at 07:33:20PM +0000, Xianwen Chen wrote:
> I am however not able to show or type Norwegian alphabet of ø, å, and
> æ. ø is shown as C8. å is shown as C%. And æ is shown is C&.

See https://www.openbsd.org/faq/faq8.html#locales. Take note of the
following:

        If logging in via xenodm(1), add export LC_CTYPE="en_US.UTF-8"
        to your ~/.xsession before starting the window manager.

Add the export command to the startup file (before invoking any window
manager) and restart X. This at least solved my problems with Norwegian
characters.

Regards,
Erling

> I tried to copi ø, å, and æ from SeaMonkey to xterm. The same thing happens.
>
> I tried to copy C8, C%, and C& from xterm back to SeaMonkey. They
> remained two characters as C8, C%, and C&.
>
> Please help.
>
> Sincerely,
> Xianwen

Reply | Threaded
Open this post in threaded view
|

Re: xterm not showing Norwegian alphabet

Xianwen Chen-3
Dear Erling,

Thank you.

I did that. I actually set both LC_ALL and LC_CTYPE.

It was helpful that you stated it worked for you. I used your
information and investigated more. It turned out that the problem was
in the shell that I use.

I use C shell, csh. When I change the shell to Bourne shell, sh, I am
able to type æ, ø, and å.

Could you check whether you can reproduce my problem by entering csh
and typing æ, ø, and å? If it is confirmed, I think I would like to
file a bug report.

Sincerely,
Xianwen

On 2/10/18, Erling Westenvik <[hidden email]> wrote:

> On Sat, Feb 10, 2018 at 07:33:20PM +0000, Xianwen Chen wrote:
>> I am however not able to show or type Norwegian alphabet of ø, å, and
>> æ. ø is shown as C8. å is shown as C%. And æ is shown is C&.
>
> See https://www.openbsd.org/faq/faq8.html#locales. Take note of the
> following:
>
> If logging in via xenodm(1), add export LC_CTYPE="en_US.UTF-8"
> to your ~/.xsession before starting the window manager.
>
> Add the export command to the startup file (before invoking any window
> manager) and restart X. This at least solved my problems with Norwegian
> characters.
>
> Regards,
> Erling
>
>> I tried to copi ø, å, and æ from SeaMonkey to xterm. The same thing
>> happens.
>>
>> I tried to copy C8, C%, and C& from xterm back to SeaMonkey. They
>> remained two characters as C8, C%, and C&.
>>
>> Please help.
>>
>> Sincerely,
>> Xianwen

Reply | Threaded
Open this post in threaded view
|

Re: xterm not showing Norwegian alphabet

Erling Westenvik-2
On Sat, Feb 10, 2018 at 10:41:19PM +0000, Xianwen Chen wrote:
> Could you check whether you can reproduce my problem by entering csh
> and typing æ, ø, and å? If it is confirmed, I think I would like to
> file a bug report.

I'm way out of my league here but I can confirm the behaviour.

(Also: when entering sh(1) from csh(1), I can type æ, ø and å again.)

Good luck!

Sincerely,
Erling

> Sincerely,
> Xianwen
>
> On 2/10/18, Erling Westenvik <[hidden email]> wrote:
> > On Sat, Feb 10, 2018 at 07:33:20PM +0000, Xianwen Chen wrote:
> >> I am however not able to show or type Norwegian alphabet of ø, å, and
> >> æ. ø is shown as C8. å is shown as C%. And æ is shown is C&.
> >
> > See https://www.openbsd.org/faq/faq8.html#locales. Take note of the
> > following:
> >
> > If logging in via xenodm(1), add export LC_CTYPE="en_US.UTF-8"
> > to your ~/.xsession before starting the window manager.
> >
> > Add the export command to the startup file (before invoking any window
> > manager) and restart X. This at least solved my problems with Norwegian
> > characters.
> >
> > Regards,
> > Erling
> >
> >> I tried to copi ø, å, and æ from SeaMonkey to xterm. The same thing
> >> happens.
> >>
> >> I tried to copy C8, C%, and C& from xterm back to SeaMonkey. They
> >> remained two characters as C8, C%, and C&.
> >>
> >> Please help.
> >>
> >> Sincerely,
> >> Xianwen

Reply | Threaded
Open this post in threaded view
|

Re: xterm not showing Norwegian alphabet

Ingo Schwarze
In reply to this post by Xianwen Chen-3
Hi Xianwen Chen,

Xianwen Chen wrote on Sat, Feb 10, 2018 at 10:41:19PM +0000:

> I use C shell, csh. When I change the shell to Bourne shell, sh,

On OpenBSD, sh(1) is not a Bourne shell, but the same binary
as ksh(1), a Korn shell, which is reasonably POSIX-compatible
with various extensions.  You can see that with ls -i.

> I am able to type [non-ASCII characters].
>
> Could you check whether you can reproduce my problem by
> entering csh and typing [non-ASCII characters].

Some time ago, i hacked into ksh(1) and added minimal UTF-8 support
to both emacs and vi input mode, with very little disruption to the
existing code base.  I didn't bother to do anything similar for csh(1);
whatever Theo may say, it is not a very modern shell IMHO...  ;)

I did not test, but i would be very surprised if csh(1) were able
to work sensibly with UTF-8 input.  Of course, i wouldn't object
to similar work being done in csh(1).

> I think I would like to file a bug report.

Please don't, it is already well-known that no UTF-8 support was
added to OpenBSD csh(1).

Yours,
  Ingo

Reply | Threaded
Open this post in threaded view
|

Re: xterm not showing Norwegian alphabet

Xianwen Chen-3
Dear Ingao,

Thank you!

I am very interested in implementing UTF-8 input into csh(1).  Would you agree that a good startig point would be to look at how your implemented UTF-8 support in ksh(1)?

I tried to find your commit on GitHub, which implement UTF-8 support in ksh(1). I could not find it. Was the change commited to a file in src/bin/ksh/? Which file was it? Do you remember the approximate time of the implementation? Such information will be helpful for me to find the implementation and to study it.

Sincerely,
Xianwen

On Sun, Feb 11, 2018 at 3:54 AM, Ingo Schwarze <[hidden email]<mailto:[hidden email]>> wrote:
Hi Xianwen Chen,

Xianwen Chen wrote on Sat, Feb 10, 2018 at 10:41:19PM +0000:

> I use C shell, csh. When I change the shell to Bourne shell, sh,

On OpenBSD, sh(1) is not a Bourne shell, but the same binary
as ksh(1), a Korn shell, which is reasonably POSIX-compatible
with various extensions.  You can see that with ls -i.

> I am able to type [non-ASCII characters].
>
> Could you check whether you can reproduce my problem by
> entering csh and typing [non-ASCII characters].

Some time ago, i hacked into ksh(1) and added minimal UTF-8 support
to both emacs and vi input mode, with very little disruption to the
existing code base.  I didn't bother to do anything similar for csh(1);
whatever Theo may say, it is not a very modern shell IMHO...  ;)

I did not test, but i would be very surprised if csh(1) were able
to work sensibly with UTF-8 input.  Of course, i wouldn't object
to similar work being done in csh(1).

> I think I would like to file a bug report.

Please don't, it is already well-known that no UTF-8 support was
added to OpenBSD csh(1).

Yours,
  Ingo

Reply | Threaded
Open this post in threaded view
|

Re: xterm not showing Norwegian alphabet

Ingo Schwarze
Hi Xianwen Chen,

Xianwen Chen wrote on Sun, Feb 11, 2018 at 08:50:20AM +0000:

> Would you agree that a good startig point would be to look at how
> your implemented UTF-8 support in ksh(1)?

Probably, that's one of the pieces of information you need.
I'm not sure it will be sufficient, though; i did not look at
the csh(1) codebase, and it may differ and pose other challenges.

In any case, make sure you study

  https://www.openbsd.org/papers/eurobsdcon2016-utf8.pdf
  https://www.openbsd.org/papers/eurobsdcon2016-utf8.roff

carefully.

> I tried to find your commit

It was more than one commit.

> on GitHub, which implement UTF-8 support in ksh(1).
> I could not find it.

I'm not surprised.  Github is simply unusable.  It is an extremely
hostile user interface.  Use

  $ cd /usr/src/bin/ksh/
  $ cvs log -wschwarze | less
  /schwarze

Some examples of relevant revisions include:

emacs.c
1.61 2015/12/10 10:00:14
1.64 2016/01/08 13:17:57
1.67 2017/05/12 14:37:52

vi.c
1.40 2016/10/11 19:52:54
1.41 2016/10/17 17:19:08
1.42 2016/10/17 17:44:47
1.47 2017/05/31 20:18:43

main.c
1.81 2016/10/11 19:52:54
1.82 2016/10/17 17:44:47

You can also use the command-line git(1) program if you understand
the bloated user interface, but simply using cvs(1) is almost
certainly easier.

The above list may not be complete, and others may also have contributed,
but most related changes are likely in emacs.c and vi.c.

> Was the change commited to a file in src/bin/ksh/?

Yes, the related changes are in that directory.

Have fun,
  Ingo