| Title | "[Errno 5] Input/output error" from Bugzilla integration logger |
| Status | closed |
| Priority | essential |
| Assigned user | Nick Barnes |
| Organization | Ravenbrook |
| Description | Two separate users running the Bugzilla integration have encountered an "Input/output error" when writing to the log. The error message is as follows: (P4DTI-8647) The replicator failed to poll successfully, because of the following problem: [Errno 5] Input/output error (P4DTI-8523) Here's a full Python traceback: [...] File "logger.py", line 113, in write self.file.write(self.format_with_date(msg)) locals: self: <logger.file_logger instance at 810e570> msg: <message.message instance at 8181b08> That is, an error was encountered when trying to write logger messages to either the standard output or to the log file. One user was running with log_level set to message.DEBUG, which will generate a lot of log output. The other user was not. |
| Analysis | It seems likely that the replicator is being run in the background of a shell, without standard output redirection, and that the shell is then exited. I can reproduce this under those circumstances. The logger should handle errors on writing to a log stream, and maybe stop logging to that stream (depending on the error). Note that the standard P4DTI startup script does redirect standard output from the replicator so will avoid this problem. The write(2) manual page on Linux says this: EIO A low-level I/O error occurred while modifying the inode. I have now been able to reproduce this, by running the replicator in the background of a shell, without standard output redirected, and exiting the parent shell. Some time later (apparently when a 4k buffer fills up in the replicator process) an EIO error is reported. I get the same results with small test programs in Python and C (C get s EIO quicker, seemingly because it doesn't have the 4k buffer). It seems to me (from the write(2) man page) that we should get EPIPE in this circumstance, but we don't. In any case, the P4DTI needs to gracefully handle errors encountered when logging. |
| How found | customer |
| Evidence | One user report: <http://info.ravenbrook.com/mail/2001/08/07/18-57-03/0.txt>Response to another user report: < http://info.ravenbrook.com/mail/2001/08/06/16-23-40/0.txt>Another user report: < http://info.ravenbrook.com/mail/2001/08/30/16-24-07/0.txt> |
| Observed in | 1.1.1 |
| Created by | Nick Barnes |
| Created on | 2001-08-08 11:17:01 |
| Last modified by | Gareth Rees |
| Last modified on | 2001-12-10 19:54:34 |
| History | 2001-08-08 NB Created. 2001-10-05 GDR Downgraded to "essential" because we have a workaround. 2001-11-20 NDL Fixed by change 24038. |
| Support | Advice for all releases. Symptom:Bugzilla integration fails and sends the following e-mail message: Diagnosis:The replicator is not being started with the P4DTI startup script, but by hand from a shell. The replicator is then placed in the background and the shell is exited. When the replicator subsequently tries to write log messages to its standard output, it cannot do so and fails with this message. Workaround: Change the way in which you invoke the replicator so that standard output is redirected to a file or to /dev/null. For instance:
(note that the standard P4DTI startup script invokes the replicator in this way). Or:
|
| Change | Effect | Date | User | Description |
|---|---|---|---|---|
| 24038 | closed | 2001-11-20 13:29:54 | Nick Levine | Error handling hook for loggers belongs to instances rather than to the logger class. |
| 24030 | closed | 2001-11-20 10:17:59 | Nick Levine | Fix job000382. |