- Not Found - The server was unable to locate the URL. Some content management systems send 404 status codes even when documents do exist. Ensure files that exist do give a 200 status code and requests for files that do not exist give a 404 status code. You may also want to check with your host to see if you can set up a custom 404 error page which makes it easy for site visitors to
- view your most popular and / or most relevant navigational options
- report navigational problems within your site
Search engines require a robots.txt file to see what portions of your site they are allowed to crawl. Many browsers request a favicon ico file when loading your site. While neither of these files are necessary, creating them will help keep your log files clean so you can focus on other errors your site might have.