HTTP response codes mnoGoSearch understands

This sections describes how mnoGoSearch processes various HTTP codes at crawling time.

• 200 OK

  1. If the -m command line argument ("force reindex") specified, GOTO 4.

  2. Comparing the new checksum with the old one stored in database

  3. If the checksum are the equal, next_index_time= Now() + Period, GOTO 7.

  4. Parsing the document, creating word list and adding all found links into the url table.

  5. Updating all information about the document in the database.

  6. Done

• 304 Not Modified

  1. next_index_time=now() + Period.

  2. Done

• 
  301 Moved Permanently
  302 Moved Temporarily
  303 See Other
        

  1. Deleting word and section information about the document.

  2. next_index_time = Now() + Period

  3. Adding the URL from the Location: header into the database.

  4. Done

• 
  300 Multiple Choices
  305 Use Proxy (proxy redirect)
  400 Bad Request
  401 Unauthorized
  402 Payment Required
  403 Forbidden
  404 Not found
  405 Method Not Allowed
  406 Not Acceptable
  407 Proxy Authentication Required
  408 Request Timeout
  409 Conflict
  410 Gone
  411 Length Required
  412 Precondition Failed
  413 Request Entity Too Large
  414 Request-URI Too Long
  415 Unsupported Media Type
  500 Internal Server Error
  501 Not Implemented
  502 Bad Gateway
  505 Protocol Version Not Supported
  

  1. Deleting all word and section information about this document.

  2. next_index_time= Now() + Period

  3. Done

• 
  503 Service Unavailable
  504 Gateway Timeout
  

  1. next_index_time= Now() + NetErrorDelayTime

  2. Done