Firefox 41.0.2 with W^X

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

Firefox 41.0.2 with W^X

Ed Ahlsen-Girard-2
I have noticed a performance hit since the switch was flipped. Firefox
stays at the top of top most of the time, and its CPU percentages have
spiked to 175% if multiple tabs were being opened. dmesg below the sig.

--

Edward Ahlsen-Girard
Ft Walton Beach, FL

OpenBSD 5.8-current (GENERIC.MP) #1537: Tue Oct 20 09:44:09 MDT 2015
    [hidden email]:/usr/src/sys/arch/amd64/compile/GENERIC.MP
real mem = 4242014208 (4045MB)
avail mem = 4109324288 (3918MB)
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.5 @ 0xf06d0 (43 entries)
bios0: vendor American Megatrends Inc. version "0504" date 10/05/2009
bios0: ASUSTeK Computer INC. P-P5G41
acpi0 at bios0: rev 2
acpi0: sleep states S0 S1 S3 S4 S5
acpi0: tables DSDT FACP APIC MCFG OEMB HPET GSCI SSDT
acpi0: wakeup devices P0P2(S4) P0P3(S4) P0P1(S4) UAR1(S4) PS2K(S4) PS2M(S4) USB0(S4) USB1(S4) USB2(S4) USB3(S4) EUSB(S4) MC97(S4) P0P4(S4) P0P5(S4) P0P6(S4) P0P7(S4) [...]
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: Intel(R) Core(TM)2 Duo CPU E7500 @ 2.93GHz, 2933.70 MHz
cpu0: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,SSE4.1,XSAVE,NXE,LONG,LAHF,PERF,SENSOR
cpu0: 3MB 64b/line 8-way L2 cache
cpu0: smt 0, core 0, package 0
mtrr: Pentium Pro MTRR support, 8 var ranges, 88 fixed ranges
cpu0: apic clock running at 266MHz
cpu0: mwait min=64, max=64, C-substates=0.2.2.2.2, IBE
cpu1 at mainbus0: apid 1 (application processor)
cpu1: Intel(R) Core(TM)2 Duo CPU E7500 @ 2.93GHz, 2933.30 MHz
cpu1: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,SSE4.1,XSAVE,NXE,LONG,LAHF,PERF,SENSOR
cpu1: 3MB 64b/line 8-way L2 cache
cpu1: smt 0, core 1, package 0
ioapic0 at mainbus0: apid 2 pa 0xfec00000, version 20, 24 pins
acpimcfg0 at acpi0 addr 0xf0000000, bus 0-63
acpihpet0 at acpi0: 14318179 Hz
acpiprt0 at acpi0: bus 0 (PCI0)
acpiprt1 at acpi0: bus -1 (P0P2)
acpiprt2 at acpi0: bus -1 (P0P3)
acpiprt3 at acpi0: bus 3 (P0P4)
acpiprt4 at acpi0: bus -1 (P0P5)
acpiprt5 at acpi0: bus 2 (P0P6)
acpiprt6 at acpi0: bus 1 (P0P7)
acpicpu0 at acpi0: !C2(500@1 mwait.1@0x10), C1(1000@1 mwait.1), PSS
acpicpu1 at acpi0: !C2(500@1 mwait.1@0x10), C1(1000@1 mwait.1), PSS
aibs0 at acpi0 RTMP RVLT RFAN GGRP GITM SITM
aibs0: FSIF: invalid package
acpibtn0 at acpi0: PWRB
cpu0: Enhanced SpeedStep 2933 MHz: speeds: 2936, 2670, 2403, 2136, 1870, 1603 MHz
pci0 at mainbus0 bus 0
pchb0 at pci0 dev 0 function 0 "Intel G41 Host" rev 0x03
vga1 at pci0 dev 2 function 0 "Intel G41 Video" rev 0x03
intagp0 at vga1
agp0 at intagp0: aperture at 0xe0000000, size 0x10000000
inteldrm0 at vga1
drm0 at inteldrm0
inteldrm0: msi
inteldrm0: 1920x1080
wsdisplay0 at vga1 mux 1: console (std, vt100 emulation)
wsdisplay0: screen 1-5 added (std, vt100 emulation)
"Intel G41 Video" rev 0x03 at pci0 dev 2 function 1 not configured
azalia0 at pci0 dev 27 function 0 "Intel 82801GB HD Audio" rev 0x01: msi
azalia0: codecs: Realtek ALC888
audio0 at azalia0
ppb0 at pci0 dev 28 function 0 "Intel 82801GB PCIE" rev 0x01: msi
pci1 at ppb0 bus 3
xhci0 at pci1 dev 0 function 0 vendor "Fresco Logic", unknown product 0x1100 rev 0x01: msi
usb0 at xhci0: USB revision 3.0
uhub0 at usb0 "Fresco Logic xHCI root hub" rev 3.00/1.00 addr 1
ppb1 at pci0 dev 28 function 2 "Intel 82801GB PCIE" rev 0x01: msi
pci2 at ppb1 bus 2
re0 at pci2 dev 0 function 0 "Realtek 8168" rev 0x02: RTL8168C/8111C (0x3c00), msi, address 48:5b:39:c5:63:95
rgephy0 at re0 phy 7: RTL8169S/8110S/8211 PHY, rev. 2
ppb2 at pci0 dev 28 function 3 "Intel 82801GB PCIE" rev 0x01: msi
pci3 at ppb2 bus 1
vendor "VIA", unknown product 0x3401 (class serial bus subclass Firewire, rev 0x00) at pci3 dev 0 function 0 not configured
vendor "VIA", unknown product 0x401a (class mass storage subclass miscellaneous, rev 0x00) at pci3 dev 0 function 1 not configured
sdhc0 at pci3 dev 0 function 2 vendor "VIA", unknown product 0x401b rev 0x00: apic 2 int 19
sdhc0 at 0x10: can't map registers
uhci0 at pci0 dev 29 function 0 "Intel 82801GB USB" rev 0x01: apic 2 int 23
uhci1 at pci0 dev 29 function 1 "Intel 82801GB USB" rev 0x01: apic 2 int 19
uhci2 at pci0 dev 29 function 2 "Intel 82801GB USB" rev 0x01: apic 2 int 18
uhci3 at pci0 dev 29 function 3 "Intel 82801GB USB" rev 0x01: apic 2 int 16
ehci0 at pci0 dev 29 function 7 "Intel 82801GB USB" rev 0x01: apic 2 int 23
usb1 at ehci0: USB revision 2.0
uhub1 at usb1 "Intel EHCI root hub" rev 2.00/1.00 addr 1
ppb3 at pci0 dev 30 function 0 "Intel 82801BA Hub-to-PCI" rev 0xe1
pci4 at ppb3 bus 4
pcib0 at pci0 dev 31 function 0 "Intel 82801GB LPC" rev 0x01
pciide0 at pci0 dev 31 function 1 "Intel 82801GB IDE" rev 0x01: DMA, channel 0 configured to compatibility, channel 1 configured to compatibility
pciide0: channel 0 disabled (no drives)
pciide0: channel 1 disabled (no drives)
pciide1 at pci0 dev 31 function 2 "Intel 82801GB SATA" rev 0x01: DMA, channel 0 configured to native-PCI, channel 1 configured to native-PCI
pciide1: using apic 2 int 19 for native-PCI interrupt
wd0 at pciide1 channel 0 drive 0: <Hitachi HDS721010CLA332>
wd0: 16-sector PIO, LBA48, 953869MB, 1953525168 sectors
atapiscsi0 at pciide1 channel 0 drive 1
scsibus1 at atapiscsi0: 2 targets
cd0 at scsibus1 targ 0 lun 0: <TEAC, DV-W524GS, BT11> ATAPI 5/cdrom removable
wd0(pciide1:0:0): using PIO mode 4, Ultra-DMA mode 6
cd0(pciide1:0:1): using PIO mode 4, Ultra-DMA mode 5
ichiic0 at pci0 dev 31 function 3 "Intel 82801GB SMBus" rev 0x01: apic 2 int 19
iic0 at ichiic0
spdmem0 at iic0 addr 0x50: 2GB DDR2 SDRAM non-parity PC2-6400CL6
spdmem1 at iic0 addr 0x52: 2GB DDR2 SDRAM non-parity PC2-6400CL6
usb2 at uhci0: USB revision 1.0
uhub2 at usb2 "Intel UHCI root hub" rev 1.00/1.00 addr 1
usb3 at uhci1: USB revision 1.0
uhub3 at usb3 "Intel UHCI root hub" rev 1.00/1.00 addr 1
usb4 at uhci2: USB revision 1.0
uhub4 at usb4 "Intel UHCI root hub" rev 1.00/1.00 addr 1
usb5 at uhci3: USB revision 1.0
uhub5 at usb5 "Intel UHCI root hub" rev 1.00/1.00 addr 1
isa0 at pcib0
isadma0 at isa0
com0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo
pckbc0 at isa0 port 0x60/5 irq 1 irq 12
pckbd0 at pckbc0 (kbd slot)
wskbd0 at pckbd0: console keyboard, using wsdisplay0
pcppi0 at isa0 port 0x61
spkr0 at pcppi0
it0 at isa0 port 0x2e/2: IT8720F rev 2, EC port 0x290
uhub0: device problem, disabling port 3
uhidev0 at uhub3 port 2 configuration 1 interface 0 "Logitech Trackball" rev 1.10/2.20 addr 2
uhidev0: iclass 3/1
ums0 at uhidev0: 3 buttons, Z dir
wsmouse0 at ums0 mux 0
vscsi0 at root
scsibus2 at vscsi0: 256 targets
softraid0 at root
scsibus3 at softraid0: 256 targets
root on wd0a (51cdc1f6f7433a99.a) swap on wd0b dump on wd0b
umass0 at uhub1 port 6 configuration 1 interface 0 "Kingston DataTraveler 2.0" rev 2.00/2.00 addr 2
umass0: using SCSI over Bulk-Only
scsibus4 at umass0: 2 targets, initiator 0
sd0 at scsibus4 targ 1 lun 0: <Kingston, DataTraveler 2.0, 1.00> SCSI2 0/direct removable serial.09511603011514254CB3
sd0: 959MB, 512 bytes/sector, 1965056 sectors
sd0 detached
scsibus4 detached
umass0 detached

Reply | Threaded
Open this post in threaded view
|

Re: Firefox 41.0.2 with W^X

David Coppa
On Thu, Oct 22, 2015 at 3:45 PM, Ed Ahlsen-Girard <[hidden email]> wrote:
> I have noticed a performance hit since the switch was flipped. Firefox
> stays at the top of top most of the time, and its CPU percentages have
> spiked to 175% if multiple tabs were being opened. dmesg below the sig.

Can you try if the attached patch is an improvement?

Ciao!
David

firefox-1215573.diff (4K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Firefox 41.0.2 with W^X

Amit Kulkarni-5
On Thu, Oct 22, 2015 at 12:26 PM, David Coppa <[hidden email]> wrote:

> On Thu, Oct 22, 2015 at 3:45 PM, Ed Ahlsen-Girard <[hidden email]>
> wrote:
> > I have noticed a performance hit since the switch was flipped. Firefox
> > stays at the top of top most of the time, and its CPU percentages have
> > spiked to 175% if multiple tabs were being opened. dmesg below the sig.
>
> Can you try if the attached patch is an improvement?
>
> Ciao!
> David
>

Hi,

This CPU spike is present with October 11 packages (Firefox 41.0.1) on
amd64, so it will be difficult to isolate the performance impact of the W
^X vs the existing situation.

Thanks
Reply | Threaded
Open this post in threaded view
|

Re: Firefox 41.0.2 with W^X

Martin Pieuchot
On 22/10/15(Thu) 21:40, Amit Kulkarni wrote:

> On Thu, Oct 22, 2015 at 12:26 PM, David Coppa <[hidden email]> wrote:
>
> > On Thu, Oct 22, 2015 at 3:45 PM, Ed Ahlsen-Girard <[hidden email]>
> > wrote:
> > > I have noticed a performance hit since the switch was flipped. Firefox
> > > stays at the top of top most of the time, and its CPU percentages have
> > > spiked to 175% if multiple tabs were being opened. dmesg below the sig.
> >
> > Can you try if the attached patch is an improvement?
> >
>
> Hi,
>
> This CPU spike is present with October 11 packages (Firefox 41.0.1) on
> amd64, so it will be difficult to isolate the performance impact of the W
> ^X vs the existing situation.

FWIW I found that firefox is (ab)using pthread_mutex_trylock(3) a lot
resulting in a storm of sched_yield(2) triggering a lot (dozen to
hundreds of thousands) of IPIs on my x220.

I tried to look at the source code but couldn't figure out where the
call of pthread_mutex_trylock(3) are coming from.  Firefox is just a
monster.

I'm sorry but I agree that if nobody is taking care of this regression
it will be really hard to measure the impact of the W^X change.

Reply | Threaded
Open this post in threaded view
|

Re: Firefox 41.0.2 with W^X

David Coppa
On Fri, Oct 23, 2015 at 2:01 PM, Martin Pieuchot <[hidden email]> wrote:

> On 22/10/15(Thu) 21:40, Amit Kulkarni wrote:
>> On Thu, Oct 22, 2015 at 12:26 PM, David Coppa <[hidden email]> wrote:
>>
>> > On Thu, Oct 22, 2015 at 3:45 PM, Ed Ahlsen-Girard <[hidden email]>
>> > wrote:
>> > > I have noticed a performance hit since the switch was flipped. Firefox
>> > > stays at the top of top most of the time, and its CPU percentages have
>> > > spiked to 175% if multiple tabs were being opened. dmesg below the sig.
>> >
>> > Can you try if the attached patch is an improvement?
>> >
>>
>> Hi,
>>
>> This CPU spike is present with October 11 packages (Firefox 41.0.1) on
>> amd64, so it will be difficult to isolate the performance impact of the W
>> ^X vs the existing situation.
>
> FWIW I found that firefox is (ab)using pthread_mutex_trylock(3) a lot
> resulting in a storm of sched_yield(2) triggering a lot (dozen to
> hundreds of thousands) of IPIs on my x220.
>
> I tried to look at the source code but couldn't figure out where the
> call of pthread_mutex_trylock(3) are coming from.  Firefox is just a
> monster.
>
> I'm sorry but I agree that if nobody is taking care of this regression
> it will be really hard to measure the impact of the W^X change.
>

Martin,

Has this problem manifested itself with firefox-41 or was it already
present with 40.x ?

ciao
David

Reply | Threaded
Open this post in threaded view
|

Re: Firefox 41.0.2 with W^X

Amit Kulkarni-5
On Mon, Nov 2, 2015 at 6:21 AM, David Coppa <[hidden email]> wrote:

> On Fri, Oct 23, 2015 at 2:01 PM, Martin Pieuchot <[hidden email]> wrote:
> > On 22/10/15(Thu) 21:40, Amit Kulkarni wrote:
> >> On Thu, Oct 22, 2015 at 12:26 PM, David Coppa <[hidden email]> wrote:
> >>
> >> > On Thu, Oct 22, 2015 at 3:45 PM, Ed Ahlsen-Girard <[hidden email]>
> >> > wrote:
> >> > > I have noticed a performance hit since the switch was flipped.
> Firefox
> >> > > stays at the top of top most of the time, and its CPU percentages
> have
> >> > > spiked to 175% if multiple tabs were being opened. dmesg below the
> sig.
> >> >
> >> > Can you try if the attached patch is an improvement?
> >> >
> >>
> >> Hi,
> >>
> >> This CPU spike is present with October 11 packages (Firefox 41.0.1) on
> >> amd64, so it will be difficult to isolate the performance impact of the
> W
> >> ^X vs the existing situation.
> >
> > FWIW I found that firefox is (ab)using pthread_mutex_trylock(3) a lot
> > resulting in a storm of sched_yield(2) triggering a lot (dozen to
> > hundreds of thousands) of IPIs on my x220.
> >
> > I tried to look at the source code but couldn't figure out where the
> > call of pthread_mutex_trylock(3) are coming from.  Firefox is just a
> > monster.
> >
> > I'm sorry but I agree that if nobody is taking care of this regression
> > it will be really hard to measure the impact of the W^X change.
> >
>
> Martin,
>
> Has this problem manifested itself with firefox-41 or was it already
> present with 40.x ?
>
> ciao
> David
>
>
Hi David,

Please see Erling Westenvik's email "wip: firefox 40" dated September 30,
2015. According to him it started in the 39-40 timeframe. It is getting so
unbearable that I have switched to iridium (thanks robert@).

Thanks
Reply | Threaded
Open this post in threaded view
|

Re: Firefox 41.0.2 with W^X

Martin Pieuchot
In reply to this post by David Coppa
On 02/11/15(Mon) 13:21, David Coppa wrote:

> On Fri, Oct 23, 2015 at 2:01 PM, Martin Pieuchot <[hidden email]> wrote:
> > On 22/10/15(Thu) 21:40, Amit Kulkarni wrote:
> >> On Thu, Oct 22, 2015 at 12:26 PM, David Coppa <[hidden email]> wrote:
> >>
> >> > On Thu, Oct 22, 2015 at 3:45 PM, Ed Ahlsen-Girard <[hidden email]>
> >> > wrote:
> >> > > I have noticed a performance hit since the switch was flipped. Firefox
> >> > > stays at the top of top most of the time, and its CPU percentages have
> >> > > spiked to 175% if multiple tabs were being opened. dmesg below the sig.
> >> >
> >> > Can you try if the attached patch is an improvement?
> >> >
> >>
> >> Hi,
> >>
> >> This CPU spike is present with October 11 packages (Firefox 41.0.1) on
> >> amd64, so it will be difficult to isolate the performance impact of the W
> >> ^X vs the existing situation.
> >
> > FWIW I found that firefox is (ab)using pthread_mutex_trylock(3) a lot
> > resulting in a storm of sched_yield(2) triggering a lot (dozen to
> > hundreds of thousands) of IPIs on my x220.
> >
> > I tried to look at the source code but couldn't figure out where the
> > call of pthread_mutex_trylock(3) are coming from.  Firefox is just a
> > monster.
> >
> > I'm sorry but I agree that if nobody is taking care of this regression
> > it will be really hard to measure the impact of the W^X change.
> >
>
> Martin,
>
> Has this problem manifested itself with firefox-41 or was it already
> present with 40.x ?

I don't remember, I can try to figure out but I'd be happy if somebody
else would do it 8)

Reply | Threaded
Open this post in threaded view
|

Re: Firefox 41.0.2 with W^X

Ted Unangst-6
In reply to this post by David Coppa
David Coppa wrote:

> On Fri, Oct 23, 2015 at 2:01 PM, Martin Pieuchot <[hidden email]> wrote:
> > On 22/10/15(Thu) 21:40, Amit Kulkarni wrote:
> >> On Thu, Oct 22, 2015 at 12:26 PM, David Coppa <[hidden email]> wrote:
> >>
> >> > On Thu, Oct 22, 2015 at 3:45 PM, Ed Ahlsen-Girard <[hidden email]>
> >> > wrote:
> >> > > I have noticed a performance hit since the switch was flipped. Firefox
> >> > > stays at the top of top most of the time, and its CPU percentages have
> >> > > spiked to 175% if multiple tabs were being opened. dmesg below the sig.
> >> >
> >> > Can you try if the attached patch is an improvement?
> >> >
> >>
> >> Hi,
> >>
> >> This CPU spike is present with October 11 packages (Firefox 41.0.1) on
> >> amd64, so it will be difficult to isolate the performance impact of the W
> >> ^X vs the existing situation.
> >
> > FWIW I found that firefox is (ab)using pthread_mutex_trylock(3) a lot
> > resulting in a storm of sched_yield(2) triggering a lot (dozen to
> > hundreds of thousands) of IPIs on my x220.
> >
> > I tried to look at the source code but couldn't figure out where the
> > call of pthread_mutex_trylock(3) are coming from.  Firefox is just a
> > monster.
> >
> > I'm sorry but I agree that if nobody is taking care of this regression
> > it will be really hard to measure the impact of the W^X change.
> >
>
> Martin,
>
> Has this problem manifested itself with firefox-41 or was it already
> present with 40.x ?

ktrace is pretty, uh, "enlightening"... This is a firefox process that's idle,
and not even displayed on screen.

Calling gettimeofday in a busy loop is probably not the most efficient means
of implementing a timeout, or whatever it's up to.

 26660 firefox  1446492775.550222 RET   gettimeofday 0
 26660 firefox  1446492775.550258 CALL  gettimeofday(0x4210c4ef548,0)
 26660 firefox  1446492775.550264 RET   gettimeofday 0
 26660 firefox  1446492775.550268 CALL  gettimeofday(0x4210c4ef538,0)
 26660 firefox  1446492775.550273 RET   gettimeofday 0
 26660 firefox  1446492775.550276 CALL  gettimeofday(0x4210c4ef538,0)
 26660 firefox  1446492775.550280 RET   gettimeofday 0
 26660 firefox  1446492775.550284 CALL  gettimeofday(0x4210c4ef548,0)
 26660 firefox  1446492775.550289 RET   gettimeofday 0
 26660 firefox  1446492775.550292 CALL  gettimeofday(0x4210c4ef548,0)
 26660 firefox  1446492775.550297 RET   gettimeofday 0
 26660 firefox  1446492775.550300 CALL  gettimeofday(0x4210c4ef538,0)
 26660 firefox  1446492775.550304 RET   gettimeofday 0
 26660 firefox  1446492775.550308 CALL  gettimeofday(0x4210c4ef548,0)
 26660 firefox  1446492775.550312 RET   gettimeofday 0
 26660 firefox  1446492775.550318 CALL  gettimeofday(0x4210c4ef528,0)
 26660 firefox  1446492775.550322 RET   gettimeofday 0
 26660 firefox  1446492775.550325 CALL  gettimeofday(0x4210c4ef548,0)
 26660 firefox  1446492775.550330 RET   gettimeofday 0
 26660 firefox  1446492775.550333 CALL  gettimeofday(0x4210c4ef538,0)
 26660 firefox  1446492775.550337 RET   gettimeofday 0
 26660 firefox  1446492775.550342 CALL  gettimeofday(0x4210c4ef548,0)
 26660 firefox  1446492775.550346 RET   gettimeofday 0
 26660 firefox  1446492775.550350 CALL  gettimeofday(0x4210c4ef538,0)
 26660 firefox  1446492775.550354 RET   gettimeofday 0
 26660 firefox  1446492775.550358 CALL  gettimeofday(0x4210c4ef538,0)
 26660 firefox  1446492775.550362 RET   gettimeofday 0
 26660 firefox  1446492775.550365 CALL  gettimeofday(0x4210c4ef568,0)
 26660 firefox  1446492775.550370 RET   gettimeofday 0
 26660 firefox  1446492775.550373 CALL  gettimeofday(0x4210c4ef538,0)
 26660 firefox  1446492775.550378 RET   gettimeofday 0
 26660 firefox  1446492775.550381 CALL  gettimeofday(0x4210c4ef538,0)
 26660 firefox  1446492775.550386 RET   gettimeofday 0
 26660 firefox  1446492775.550389 CALL  gettimeofday(0x4210c4ef538,0)
 26660 firefox  1446492775.550393 RET   gettimeofday 0
 26660 firefox  1446492775.550397 CALL  gettimeofday(0x4210c4ef538,0)
 26660 firefox  1446492775.550401 RET   gettimeofday 0
 26660 firefox  1446492775.550404 CALL  gettimeofday(0x4210c4ef538,0)
 26660 firefox  1446492775.550409 RET   gettimeofday 0
 26660 firefox  1446492775.550412 CALL  gettimeofday(0x4210c4ef538,0)
 26660 firefox  1446492775.550416 RET   gettimeofday 0
 26660 firefox  1446492775.550420 CALL  gettimeofday(0x4210c4ef538,0)
 26660 firefox  1446492775.550424 RET   gettimeofday 0
 26660 firefox  1446492775.550428 CALL  gettimeofday(0x4210c4ef528,0)
 26660 firefox  1446492775.550432 RET   gettimeofday 0
 26660 firefox  1446492775.550435 CALL  gettimeofday(0x4210c4ef548,0)
 26660 firefox  1446492775.550440 RET   gettimeofday 0
 26660 firefox  1446492775.550443 CALL  gettimeofday(0x4210c4ef568,0)
 26660 firefox  1446492775.550447 RET   gettimeofday 0
 26660 firefox  1446492775.550451 CALL  gettimeofday(0x4210c4ef498,0)
 26660 firefox  1446492775.550455 RET   gettimeofday 0
 26660 firefox  1446492775.550459 CALL  gettimeofday(0x4210c4ef4a8,0)
 26660 firefox  1446492775.550463 RET   gettimeofday 0
 26660 firefox  1446492775.550467 CALL  gettimeofday(0x4210c4ef528,0)
 26660 firefox  1446492775.550471 RET   gettimeofday 0
 26660 firefox  1446492775.550474 CALL  gettimeofday(0x4210c4ef568,0)
 26660 firefox  1446492775.550478 RET   gettimeofday 0
 26660 firefox  1446492775.550482 CALL  gettimeofday(0x4210c4ef498,0)
 26660 firefox  1446492775.550486 RET   gettimeofday 0
 26660 firefox  1446492775.550507 CALL  gettimeofday(0x4210c4ef4a8,0)
 26660 firefox  1446492775.550512 RET   gettimeofday 0
 26660 firefox  1446492775.550515 CALL  gettimeofday(0x4210c4ef528,0)
 26660 firefox  1446492775.550519 RET   gettimeofday 0
 26660 firefox  1446492775.550523 CALL  gettimeofday(0x4210c4ef568,0)