awk(1) rshift/lshift on the borders of int

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

awk(1) rshift/lshift on the borders of int

Martin Pelikan
I can't say anything more than 'this solved my problem', which was

$ echo | awk '{ printf "%u\n", rshift(int("0xdeffffff"), 24) }' | bc -e
'obase=16'
FFFFFF80

vs.

$ echo | awk '{ printf "%u\n", rshift(int("0xdeffffff"), 24) }' | bc -e
'obase=16'    
DE

I hope it starts a discussion (at least), but I don't really know what
the correct way is to these problems.

--
Martin Pelikan


Index: run.c
===================================================================
RCS file: /cvs/src/usr.bin/awk/run.c,v
retrieving revision 1.33
diff -u -p -r1.33 run.c
--- run.c 28 Sep 2011 19:27:18 -0000 1.33
+++ run.c 17 Jan 2012 15:01:32 -0000
@@ -1562,7 +1562,7 @@ Cell *bltin(Node **a, int n) /* builtin
  break;
  }
  y = execute(a[1]->nnext);
- u = ((int)getfval(x)) << ((int)getfval(y));
+ u = ((long long)getfval(x)) << ((long long)getfval(y));
  tempfree(y);
  nextarg = nextarg->nnext;
  break;
@@ -1573,7 +1573,7 @@ Cell *bltin(Node **a, int n) /* builtin
  break;
  }
  y = execute(a[1]->nnext);
- u = ((int)getfval(x)) >> ((int)getfval(y));
+ u = ((long long)getfval(x)) >> ((long long)getfval(y));
  tempfree(y);
  nextarg = nextarg->nnext;
  break;

Reply | Threaded
Open this post in threaded view
|

Re: awk(1) rshift/lshift on the borders of int

Jakub Tuček/otaznik-2
Martin Pelikan <[hidden email]> napsal:

> I can't say anything more than 'this solved my problem', which was
>
> $ echo | awk '{ printf "%u\n", rshift(int("0xdeffffff"), 24) }' | bc -e
> 'obase=16'
> FFFFFF80
> vs.
> $ echo | awk '{ printf "%u\n", rshift(int("0xdeffffff"), 24) }' | bc -e
> 'obase=16'    
> DE
>
> I hope it starts a discussion (at least), but I don't really know what
> the correct way is to these problems.
>
> --
> Martin Pelikan
>

Hello Martin,

You reached int boundary, so You received minimal number possible. I do not think extending boundary is good solution (that way you will reach it soon again).
To demonstrate:
$ awk 'BEGIN { printf("%d\n","0x7fffffff") }'  
2147483647

$ awk 'BEGIN { printf("%d\n","0x80000000") }'  
$ awk 'BEGIN { printf("%d\n","0x80000001") }'  
$ awk 'BEGIN { printf("%d\n","0xffffffff") }'  
$ awk 'BEGIN { printf("%d\n","0xfeedbeef") }'  
$ awk 'BEGIN { printf("%d\n","0xfffffffffffff") }'
-2147483648

In my point of view, You should solve the problem elsehow. For example splitting that big number to smaller or just trim last six characters (it depends on problem you have)

Best regards
--
Jakub TuD
ek/otaznik <[hidden email]>