Help

Troubleshooting

Log files

The system log file is an extremely useful resource for tracking down errors. This can be activated on the admin/log configuration page. A log level setting of LOGGER_DEBUG is preferred for stable production sites. Most problems with communication or storage are listed here. A setting of LOGGER_DATA provides much more detail, but can fill up your hard drive. In any case, we recommend using logrotate on your operating system to check the logs and delete older entries.

At the end of the .htconfig.php file there are several (commented out) lines that enable PHP error logging. This reports problems with the code syntax and execution of the code and is the first place you should look for problems that result in a ‘white screen’ or blank page. This is usually the result of code/syntax problems. Database errors are reported in the system log file, but we have found it useful to create a file called dbfail.out in your root directory where only database related problems are collected. If the file exists and is writable, database errors will be logged in this file as well as in the system log file.

In the case of ‘500’ errors, the problems are often logged in the logs of your web server, often in /var/log/apache2/error.log or similar. Consult the documentation of your operating system. There are three different logging options.

The first is the database error log.

This is only used if you create a file with the name dbfail.out in the root directory of your website and make it writable for the web server. If database queries have failed, they are reported here. They generally indicate typing errors in our queries, but also occur when the database server interrupts the connection or tables become corrupted. In rare cases, race conditions are reported here when two processes try to create an xchan or cache entry with the same ID. All other errors (especially persistent errors) should be investigated.

The second area is the PHP error log.

This is generated by the language processor and only reports problems in the language environment. Again, these can be syntax or programming errors, but they are usually fatal and result in a ‘white screen of death’, i.e. PHP is terminated. You should look at this file if something goes wrong that does not result in a white screen, but it is not uncommon for this file to be blank for days.

At the end of the supplied .htconfig.php file are some lines that, if not commented out, will enable a PHP error log (extremely useful for finding the cause of white screen errors). This is not the case by default as there may be issues with log file ownership and write permissions and the log file does not rotate by default.

The third file is the ‘application log’.

This is used by Hubzilla to report what is going on in the programme and usually reports any difficulties or unexpected data we have received. Occasionally, ‘heartbeat’ status messages are also output to indicate that we have reached a certain point in a script. This is the most important log file for us, as we create it ourselves for the sole purpose of reporting the status of background tasks and anything that seems strange or out of place. It doesn't necessarily have to be something bad, but maybe just unexpected. If you are running a task and a problem occurs, let us know what is in this file when the problem occurs. (Please don't send me 100M dumps, that would just annoy me). Just a few relevant lines so I can eliminate a few hundred thousand lines of code and focus on where the problem is occurring.

These are your website logs, not mine. We report serious issues at every log level. For most websites, I recommend the DEBUG log level - it provides a little extra information and doesn't generate huge log files. If a problem occurs that defies all attempts to track it, you should use the DATA log level for a short period of time to capture all the details of the structures we were dealing with at the time. This log level consumes a lot of memory and is therefore only recommended for short periods of time or for developer test sites. I recommend configuring logrotate for both the php log and the application log. I usually take a look at dbfail.out every week or two, fix any reported issues and then start over with a new file. The same goes for the PHP log file. I check it from time to time to see if there is anything that needs to be fixed.

If something goes wrong and it's not a serious error, I look at the application log file. Often I will run

tail -f logfile.out

while repeating a process where there are problems. Often I'll add additional logging statements to the code if there's no indication of what's going wrong. Even something as simple as ‘got here’ or expressing the value of a variable that might be suspicious. You can do this too - in fact, I would encourage you to do so. Once you've found what you need to look for, you can run

git checkout file.php

to immediately delete any extra logging data you've added. Use the information from this log and any details you can provide from your investigation of the problem to file your bug report - unless your analysis points to the source of the problem. In this case, simply fix it.

Rotating log files

1. Activate the logrot addon in the official hubzilla-addons repo
2. Create a directory in your web root called log with write permissions for the web server
3. Go to the logrot admin settings and enter this folder name and the maximum size and number of log files kept.

Report problems

When reporting problems, please try to provide as much detail as necessary to allow the developers to reproduce the problem and include the full text of any error messages.

We encourage you to use these logs, along with the source code you have, to the best of your ability to fix problems and find their cause. The community can often help, but only you have access to your site's log files, and it is considered a security risk to share them.

If a code issue has been uncovered, please report it in the project's bug tracker (https://framagit.org/hubzilla/core/issues). Again, provide as much detail as possible so we don't have to keep asking questions about your configuration or duplicating the issue so we can address the problem directly and figure out what to do. You are also welcome to offer your own solutions and submit patches. In fact, we encourage you to do so, as we are all volunteers and have limited time available. The more people that help out, the easier the work will be for everyone. It's okay if your solution isn't perfect. Every little bit helps, and maybe we can improve it.