understanding ldapd log error messages

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

understanding ldapd log error messages

Predrag Punosevac-2
Hi misc,

ldapd on one of my two ldap servers stop working overnight

# uname -a                                                              
OpenBSD atlas.int.autonlab.org 6.1 GENERIC.MP#20 amd64

I manually restarted it and it appears to work OK.  I started digging
little bit through the log files and I see the following in my
/var/log/message file before and after the upgrade. Can somebody help me
understand what I see and point me to some kind reading? Is my database
corrupted for some reason? The LDAP server overall has being working
really well for the past 3.5 years with Red Hat, FreeBSD, and OpenBSD
clients. I have being upgrading this server since the last database
format change (I think OpenBSD 5.3 or 5.4).

Best,
Predrag




Apr 17 04:03:39 atlas ldapd[83666]: indexed key [uid=emcfowla,ou=users,]
doesn't exist!
Apr 17 04:03:39 atlas ldapd[83666]: indexed key [uid=seohyuns,ou=users,]
doesn't exist!
Apr 17 04:03:44 atlas ldapd[83666]: error 0x24 on connection 17
Apr 17 04:04:20 atlas last message repeated 3 times
Apr 17 04:04:31 atlas ldapd[83666]: error 0x24 on connection 13
Apr 17 04:05:02 atlas last message repeated 3 times
Apr 17 04:05:12 atlas ldapd[83666]: error 0x24 on connection 13
Apr 17 04:05:24 atlas ldapd[83666]: error 0x24 on connection 18
Apr 17 04:05:57 atlas last message repeated 3 times
Apr 17 04:08:08 atlas last message repeated 12 times
Apr 17 04:08:41 atlas last message repeated 3 times
Apr 17 04:08:53 atlas ldapd[83666]: error 0x24 on connection 20
Apr 17 04:09:05 atlas ldapd[83666]: error 0x24 on connection 22
Apr 17 04:09:36 atlas last message repeated 3 times
Apr 17 04:09:57 atlas last message repeated 2 times
Apr 17 04:10:07 atlas ldapd[83666]: error 0x24 on connection 38
Apr 17 04:10:18 atlas ldapd[83666]: error 0x24 on connection 25
Apr 17 04:10:28 atlas ldapd[83666]: error 0x24 on connection 25
Apr 17 04:10:40 atlas ldapd[83666]: error 0x24 on connection 38
Apr 17 04:10:52 atlas ldapd[83666]: error 0x24 on connection 40
Apr 17 04:11:23 atlas last message repeated 3 times
Apr 17 04:11:34 atlas ldapd[83666]: error 0x24 on connection 26
Apr 17 04:11:39 atlas ldapd[83666]: indexed key [uid=dwang,ou=users,]
doesn't exist!
Apr 17 04:11:39 atlas ldapd[83666]: indexed key [uid=emcfowla,ou=users,]
doesn't exist!
Apr 17 04:11:39 atlas ldapd[83666]: indexed key [uid=seohyuns,ou=users,]
doesn't exist!
Apr 17 04:11:44 atlas ldapd[83666]: error 0x24 on connection 40
Apr 17 04:11:56 atlas ldapd[83666]: error 0x24 on connection 41
Apr 17 04:12:29 atlas last message repeated 3 times
Apr 17 04:12:50 atlas last message repeated 2 times
Apr 17 04:13:00 atlas ldapd[83666]: error 0x24 on connection 28
Apr 17 04:13:36 atlas last message repeated 3 times
Apr 17 04:14:09 atlas last message repeated 3 times
Apr 17 04:14:21 atlas ldapd[83666]: error 0x24 on connection 14
Apr 17 04:14:44 atlas last message repeated 2 times
Apr 17 04:14:54 atlas ldapd[83666]: error 0x24 on connection 15
Apr 17 04:15:25 atlas last message repeated 3 times
Apr 17 04:16:10 atlas last message repeated 4 times
Apr 17 04:16:20 atlas ldapd[83666]: error 0x24 on connection 29
Apr 17 04:16:53 atlas last message repeated 3 times
Apr 17 04:17:05 atlas ldapd[83666]: error 0x24 on connection 29
Apr 17 04:17:17 atlas ldapd[83666]: error 0x24 on connection 32
Apr 17 04:17:50 atlas last message repeated 3 times
Apr 17 04:18:23 atlas last message repeated 3 times
Apr 17 04:18:33 atlas ldapd[83666]: error 0x24 on connection 16
Apr 17 04:19:06 atlas last message repeated 3 times
Apr 17 04:19:18 atlas ldapd[83666]: error 0x24 on connection 33
Apr 17 04:19:51 atlas last message repeated 3 times
Apr 17 04:20:01 atlas ldapd[83666]: error 0x24 on connection 33
Apr 17 04:20:11 atlas ldapd[83666]: indexed key [uid=dwang,ou=users,]
doesn't exist!
Apr 17 04:20:11 atlas ldapd[83666]: indexed key [uid=emcfowla,ou=users,]
doesn't exist!
Apr 17 04:20:11 atlas ldapd[83666]: indexed key [uid=seohyuns,ou=users,]
doesn't exist!
Apr 17 04:20:12 atlas ldapd[83666]: error 0x24 on connection 34
Apr 17 04:20:22 atlas ldapd[83666]: error 0x24 on connection 13
Apr 17 04:20:33 atlas ldapd[83666]: error 0x24 on connection 34

and this is from today
Apr 21 12:04:41 atlas ldapd[78718]: error 0x24 on connection 19
Apr 21 12:04:52 atlas ldapd[78718]: error 0x24 on connection 19
Apr 21 12:04:59 atlas ldapd[78718]: indexed key [uid=dwang,ou=users,]
doesn't exist!
Apr 21 12:04:59 atlas ldapd[78718]: indexed key [uid=emcfowla,ou=users,]
doesn't exist!
Apr 21 12:04:59 atlas ldapd[78718]: indexed key [uid=seohyuns,ou=users,]
doesn't exist!
Apr 21 12:05:00 atlas ldapd[78718]: indexed key [uid=dwang,ou=users,]
doesn't exist!
Apr 21 12:05:00 atlas ldapd[78718]: indexed key [uid=emcfowla,ou=users,]
doesn't exist!
Apr 21 12:05:00 atlas ldapd[78718]: indexed key [uid=seohyuns,ou=users,]
doesn't exist!
Apr 21 12:05:01 atlas ldapd[78718]: indexed key [uid=dwang,ou=users,]
doesn't exist!
Apr 21 12:05:01 atlas ldapd[78718]: indexed key [uid=emcfowla,ou=users,]
doesn't exist!
Apr 21 12:05:01 atlas ldapd[78718]: indexed key [uid=seohyuns,ou=users,]
doesn't exist!
Apr 21 12:05:02 atlas ldapd[78718]: error 0x24 on connection 19
Apr 21 12:05:33 atlas last message repeated 3 times
Apr 21 12:07:11 atlas last message repeated 9 times
Apr 21 12:07:22 atlas ldapd[78718]: error 0x24 on connection 21
Apr 21 12:07:53 atlas last message repeated 3 times
Apr 21 12:08:46 atlas last message repeated 5 times
Apr 21 12:08:58 atlas ldapd[78718]: error 0x24 on connection 22
Apr 21 12:09:29 atlas last message repeated 3 times
Apr 21 12:11:27 atlas last message repeated 11 times
Apr 21 12:11:35 atlas ldapd[78718]: indexed key [uid=dwang,ou=users,]
doesn't exist!
Apr 21 12:11:35 atlas ldapd[78718]: indexed key [uid=emcfowla,ou=users,]
doesn't exist!
Apr 21 12:11:35 atlas ldapd[78718]: indexed key [uid=seohyuns,ou=users,]
doesn't exist!
Apr 21 12:11:37 atlas ldapd[78718]: error 0x24 on connection 23
Apr 21 12:12:01 atlas last message repeated 2 times
Apr 21 12:13:29 atlas last message repeated 8 times
Apr 21 12:13:40 atlas ldapd[78718]: error 0x24 on connection 26
Apr 21 12:14:11 atlas last message repeated 3 times
Apr 21 12:15:16 atlas last message repeated 6 times
Apr 21 12:15:28 atlas ldapd[78718]: error 0x24 on connection 28
Apr 21 12:15:49 atlas last message repeated 2 times
Apr 21 12:15:59 atlas ldapd[78718]: error 0x24 on connection 13
Apr 21 12:16:32 atlas last message repeated 3 times
Apr 21 12:17:15 atlas last message repeated 4 times
Apr 21 12:17:15 atlas ldapd[78718]: indexed key [uid=dwang,ou=users,]
doesn't exist!
Apr 21 12:17:15 atlas ldapd[78718]: indexed key [uid=emcfowla,ou=users,]
doesn't exist!
Apr 21 12:17:15 atlas ldapd[78718]: indexed key [uid=seohyuns,ou=users,]
doesn't exist!
Apr 21 12:17:16 atlas ldapd[78718]: indexed key [uid=dwang,ou=users,]
doesn't exist!
Apr 21 12:17:16 atlas ldapd[78718]: indexed key [uid=emcfowla,ou=users,]
doesn't exist!
Apr 21 12:17:16 atlas ldapd[78718]: indexed key [uid=seohyuns,ou=users,]
doesn't exist!
Apr 21 12:17:27 atlas ldapd[78718]: error 0x24 on connection 28
Apr 21 12:18:00 atlas last message repeated 3 times
Apr 21 12:18:10 atlas ldapd[78718]: error 0x24 on connection 30
Apr 21 12:18:31 atlas last message repeated 2 times
Apr 21 12:18:43 atlas ldapd[78718]: error 0x24 on connection 15
Apr 21 12:18:55 atlas ldapd[78718]: error 0x24 on connection 31
Apr 21 12:19:07 atlas ldapd[78718]: error 0x24 on connection 31
Apr 21 12:19:16 atlas ldapd[78718]: error 0x24 on connection 16
Apr 21 12:19:26 atlas ldapd[78718]: error 0x24 on connection 18
Apr 21 12:19:57 atlas last message repeated 3 times
Apr 21 12:20:08 atlas ldapd[78718]: error 0x24 on connection 31



Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: understanding ldapd log error messages

Predrag Punosevac-2
Predrag Punosevac write:
> Hi misc,
>
> ldapd on one of my two ldap servers stop working overnight
>

ldapd died again overnight. I noticed that this started happening not
right after the upgrade to 6.1 but less than 24h  after I added a
person to my LDAP database. How do I go about debugging a daemon? I am
reading

http://man.openbsd.org/rc.d

and I have used option -d when a daemon fails to start but I really need
to catch what happens when ldapd dies and redirect to the log file.

Best,
Predrag



> # uname -a                                                              
> OpenBSD atlas.int.autonlab.org 6.1 GENERIC.MP#20 amd64
>
> I manually restarted it and it appears to work OK.  I started digging
> little bit through the log files and I see the following in my
> /var/log/message file before and after the upgrade. Can somebody help me
> understand what I see and point me to some kind reading? Is my database
> corrupted for some reason? The LDAP server overall has being working
> really well for the past 3.5 years with Red Hat, FreeBSD, and OpenBSD
> clients. I have being upgrading this server since the last database
> format change (I think OpenBSD 5.3 or 5.4).
>
> Best,
> Predrag

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: understanding ldapd log error messages

Robert Klein
Hi,

On Sat, 22 Apr 2017 21:55:58 -0400
Predrag Punosevac <[hidden email]> wrote:

> Predrag Punosevac write:
> > Hi misc,
> >
> > ldapd on one of my two ldap servers stop working overnight
> >  
>
> ldapd died again overnight. I noticed that this started happening not
> right after the upgrade to 6.1 but less than 24h  after I added a
> person to my LDAP database. How do I go about debugging a daemon? I am
> reading
>
> http://man.openbsd.org/rc.d
>
> and I have used option -d when a daemon fails to start but I really
> need to catch what happens when ldapd dies and redirect to the log
> file.


Use the options "-dv" at first.  If you need to see th BER messages use
"-dvv"  (see also "man ldapd").

Could you post an example setup, i.e. ldapd.conf and a LDIF file?

Best regards
Robert

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: understanding ldapd log error messages

Predrag Punosevac-2
Robert Klein <[hidden email]> wrote:

> Hi,
>
> On Sat, 22 Apr 2017 21:55:58 -0400
> Predrag Punosevac <[hidden email]> wrote:
>
> > Predrag Punosevac write:
> > > Hi misc,
> > >
> > > ldapd on one of my two ldap servers stop working overnight
> > >  
> >
> > ldapd died again overnight. I noticed that this started happening not
> > right after the upgrade to 6.1 but less than 24h  after I added a
> > person to my LDAP database. How do I go about debugging a daemon? I am
> > reading
> >
> > http://man.openbsd.org/rc.d
> >
> > and I have used option -d when a daemon fails to start but I really
> > need to catch what happens when ldapd dies and redirect to the log
> > file.
>
>
> Use the options "-dv" at first.  If you need to see th BER messages use
> "-dvv"  (see also "man ldapd").
>
> Could you post an example setup, i.e. ldapd.conf and a LDIF file?

# more /etc/ldapd.conf
#       $OpenBSD: ldapd.conf,v 1.2 2010/06/29 02:50:22 martinh Exp $

schema "/etc/ldap/core.schema"
schema "/etc/ldap/inetorgperson.schema"
schema "/etc/ldap/nis.schema"

listen on lo0 tls certificate atlas
listen on em1 tls certificate atlas
listen on "/var/run/ldapi"

namespace "dc=autonlab,dc=org" {
        rootdn          "cn=admin,dc=autonlab,dc=org"
        rootpw          "{SSHA}iV3eDxcQ9LM9EJN6ltigbmHFUwuS/tE/"
        index           sn
        index           givenName
        index           cn
        index           mail
}


This is an example of newuser.ldif file used to add new users to the
database.  Note the following file is sanitized for trailing white
spaces. The white spaces you see in my e-mail are not in the database.

# more new_user.ldif
dn: cn=jsmith,ou=group,dc=autonlab,dc=org
cn: jsmith
objectClass: top
objectClass: posixGroup
gidNumber: 1120
memberUid: jsmith
description: User Private Group


dn: uid=jsmith,ou=users,dc=autonlab,dc=org
uid: jsmith
cn: John Smith
sn: Smith
givenName: John
displayName: John Smith
objectClass: inetOrgPerson
objectClass: posixAccount
objectClass: top
objectClass: shadowAccount
shadowLastChange: 1492716996
userPassword: {SSHA}E7VQcALE0zXe4lehOulF/fXIdi2kUQ6b
shadowMin: 1
shadowMax: 180
shadowWarning: 7
shadowInactive: 30
shadowExpire: -1
shadowFlag: 0
loginShell: /bin/bash
uidNumber: 1120
gidNumber: 1120
homeDirectory: /zfsauton/home/jsmith
mail: [hidden email]
gecos: John Smith
title: MSc student
postalAddress: NSH 3128
postalAddress: CMU
businessCategory: Graduate Student
telephoneNumber: (412) ???-????
o: Auton Lab




>
> Best regards
> Robert

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: understanding ldapd log error messages

Predrag Punosevac-2
In reply to this post by Predrag Punosevac-2
> Use the options "-dv" at first.  If you need to see th BER messages
> use "-dvv"  (see also "man ldapd").

Do you see anything in this snipped

b,dc=org by any, in namespace dc=autonlab,dc=org
Apr 23 23:19:09.481 [18682] found dn
uid=rrabbany,ou=users,dc=autonlab,dc=org
Apr 23 23:19:09.481 [18682] requesting 01 access to
uid=rrabbany,ou=users,dc=autonlab,dc=org by any, in namespace
dc=autonlab,dc=org
Apr 23 23:19:09.481 [18682] found dn
uid=zhez,ou=users,dc=autonlab,dc=org
Apr 23 23:19:09.481 [18682] requesting 01 access to
uid=zhez,ou=users,dc=autonlab,dc=org by any, in namespace
dc=autonlab,dc=org
Apr 23 23:19:09.482 [18682] found dn
uid=awertz,ou=users,dc=autonlab,dc=org
Apr 23 23:19:09.482 [18682] requesting 01 access to
uid=awertz,ou=users,dc=autonlab,dc=org by any, in namespace
dc=autonlab,dc=org
Apr 23 23:19:09.482 [18682] 259 scanned, 135 matched, 0 dups
Apr 23 23:19:09.482 [18682] sending response 5 with result 0
Apr 23 23:19:09.482 [18682] finished search on msgid 3
Apr 23 23:19:09.645 [18682] end-of-file on connection 16
Apr 23 23:19:09.645 [18682] closing connection 16
Apr 23 23:19:10.960 [18682] accepted connection from 10.8.0.6 on fd 16
Apr 23 23:19:10.961 [18682] consumed 31 bytes
Apr 23 23:19:10.961 [18682] got request type 23, id 1
Apr 23 23:19:10.961 [18682] got extended operation
1.3.6.1.4.1.1466.20037
Apr 23 23:19:10.961 [18682] sending response 24 with result 0
Apr 23 23:19:10.961 [18682] conn_tls_init: switching to TLS
Apr 23 23:19:10.978 [18682] error 0x24 on connection 16
Apr 23 23:19:10.978 [18682] closing connection 16




Apr 23 23:19:30.086 [18682] consumed 31 bytes
Apr 23 23:19:30.086 [18682] got request type 23, id 1
Apr 23 23:19:30.086 [18682] got extended operation
1.3.6.1.4.1.1466.20037
Apr 23 23:19:30.086 [18682] sending response 24 with result 0
Apr 23 23:19:30.086 [18682] conn_tls_init: switching to TLS
Apr 23 23:19:30.105 [18682] error 0x24 on connection 16
Apr 23 23:19:30.105 [18682] closing connection 16
Apr 23 23:19:41.811 [18682] accepted connection from 10.8.0.6 on fd 16
Apr 23 23:19:41.811 [18682] consumed 31 bytes
Apr 23 23:19:41.811 [18682] got request type 23, id 1
Apr 23 23:19:41.811 [18682] got extended operation
1.3.6.1.4.1.1466.20037
Apr 23 23:19:41.811 [18682] sending response 24 with result 0
Apr 23 23:19:41.811 [18682] conn_tls_init: switching to TLS
Apr 23 23:19:41.828 [18682] error 0x24 on connection 16
Apr 23 23:19:41.828 [18682] closing connection 16
Apr 23 23:19:51.312 [18682] accepted connection from 10.8.0.6 on fd 16
Apr 23 23:19:51.312 [18682] consumed 31 bytes
Apr 23 23:19:51.312 [18682] got request type 23, id 1
Apr 23 23:19:51.312 [18682] got extended operation
1.3.6.1.4.1.1466.20037
Apr 23 23:19:51.312 [18682] sending response 24 with result 0
Apr 23 23:19:51.312 [18682] conn_tls_init: switching to TLS
Apr 23 23:19:51.331 [18682] error 0x24 on connection 16
Apr 23 23:19:51.331 [18682] closing connection 16

Apr 23 23:20:26.243 [18682] requesting 01 access to
uid=yichongx,ou=users,dc=autonlab,dc=org by any, in namespace
dc=autonlab,dc=org
Apr 23 23:20:26.243 [18682] found dn
uid=sray,ou=users,dc=autonlab,dc=org
Apr 23 23:20:26.243 [18682] requesting 01 access to
uid=sray,ou=users,dc=autonlab,dc=org by any, in namespace
dc=autonlab,dc=org
Apr 23 23:20:26.244 [18682] found dn
uid=ekennedy,ou=users,dc=autonlab,dc=org
Apr 23 23:20:26.244 [18682] requesting 01 access to
uid=ekennedy,ou=users,dc=autonlab,dc=org by any, in namespace
dc=autonlab,dc=org
Apr 23 23:20:26.244 [18682] found dn
uid=aharley,ou=users,dc=autonlab,dc=org
Apr 23 23:20:26.244 [18682] requesting 01 access to
uid=aharley,ou=users,dc=autonlab,dc=org by any, in namespace
dc=autonlab,dc=org
Apr 23 23:20:26.244 [18682] found dn
uid=rrabbany,ou=users,dc=autonlab,dc=org
Apr 23 23:20:26.244 [18682] requesting 01 access to
uid=rrabbany,ou=users,dc=autonlab,dc=org by any, in namespace
dc=autonlab,dc=org
Apr 23 23:20:26.244 [18682] found dn
uid=zhez,ou=users,dc=autonlab,dc=org
Apr 23 23:20:26.244 [18682] requesting 01 access to
uid=zhez,ou=users,dc=autonlab,dc=org by any, in namespace
dc=autonlab,dc=org
Apr 23 23:20:26.245 [18682] found dn
uid=awertz,ou=users,dc=autonlab,dc=org
Apr 23 23:20:26.245 [18682] requesting 01 access to
uid=awertz,ou=users,dc=autonlab,dc=org by any, in namespace
dc=autonlab,dc=org
Apr 23 23:20:26.245 [18682] 259 scanned, 0 matched, 0 dups
Apr 23 23:20:26.245 [18682] sending response 5 with result 0
Apr 23 23:20:26.245 [18682] finished search on msgid 7



Loading...