pathological behavior in fread

classic Classic list List threaded Threaded
2 messages Options
enh
Reply | Threaded
Open this post in threaded view
|

pathological behavior in fread

enh

this code takes minutes to run on a high-end desktop when using the OpenBSD fread:

  FILE* fp = fopen("/dev/zero", "r");
  ASSERT_TRUE(fp != NULL);
  setvbuf(fp, 0, _IONBF, 0);
  char buf[64*1024];
  for (size_t i = 0; i < 1024; ++i) {
    fread(buf, 64*1024, 1, fp);
  }
  fclose(fp);

here's a patch to avoid interpreting "unbuffered" as "buffer one byte at a time":

--- a/libc/upstream-openbsd/lib/libc/stdio/fread.c
+++ b/libc/upstream-openbsd/lib/libc/stdio/fread.c
@@ -68,6 +68,22 @@ fread(void *buf, size_t size, size_t count, FILE *fp)
  fp->_r = 0;
  total = resid;
  p = buf;
+
+ // BEGIN android-added
+ // Avoid pathological behavior on unbuffered files. OpenBSD
+ // will loop reading one byte then memcpying one byte!
+ if ((fp->_flags & __SNBF) != 0) {
+ // We know if we're unbuffered that our buffer is empty, so
+ // we can just read directly.
+ while (resid > 0 && (r = (*fp->_read)(fp->_cookie, p, resid)) > 0) {
+ p += r;
+ resid -= r;
+ }
+ FUNLOCKFILE(fp);
+ return ((total - resid) / size);
+ }
+ // END android-added
+
  while (resid > (r = fp->_r)) {
  (void)memcpy((void *)p, (void *)fp->_p, (size_t)r);
  fp->_p += r;

i suspect we should rewrite the regular case to avoid the intermediate buffer
for large reads, but this is about all i feel brave enough to try in a point
release.

 --elliott

Reply | Threaded
Open this post in threaded view
|

Re: pathological behavior in fread

Ted Unangst-6
On Mon, Dec 01, 2014 at 16:22, Elliott Hughes wrote:
>
> this code takes minutes to run on a high-end desktop when using the
> OpenBSD fread:

Yikes. Thanks for the patch. Here's a version for OpenBSD.

Index: stdio/fread.c
===================================================================
RCS file: /cvs/src/lib/libc/stdio/fread.c,v
retrieving revision 1.12
diff -u -p -r1.12 fread.c
--- stdio/fread.c 1 May 2014 16:40:36 -0000 1.12
+++ stdio/fread.c 2 Dec 2014 08:02:37 -0000
@@ -68,6 +68,21 @@ fread(void *buf, size_t size, size_t cou
  fp->_r = 0;
  total = resid;
  p = buf;
+
+ if ((fp->_flags & __SNBF) != 0) {
+ /*
+ * We know if we're unbuffered that our buffer is empty, so
+ * we can just read directly. This is much faster than the
+ * loop below which will perform a series of one byte reads.
+ */
+ while (resid > 0 && (r = (*fp->_read)(fp->_cookie, p, resid)) > 0) {
+ p += r;
+ resid -= r;
+ }
+ FUNLOCKFILE(fp);
+ return ((total - resid) / size);
+ }
+
  while (resid > (r = fp->_r)) {
  (void)memcpy((void *)p, (void *)fp->_p, (size_t)r);
  fp->_p += r;