CustoSec:Check Http Services Enh

From CustosecWiki
Jump to navigation Jump to search
caption
Basic Information on Check
Name of Check HTTP Services Enhanced Technical Name check_http_service
Available in Plugin Extended Web Server Monitoring (€) Number of Arguments 15
From Version ARANSEC 2.41 Compability > ARANSEC 2.40




Scope of Check

This is a comprehensive check with a large number of arguments to monitor almost everything on a web server. With it's widely enhanced script architecture it can also be used to access web servers through proxy servers and use standard or ssh-connections.

The check can query information in the html header of a page as well as in the body (content) of a page. A status wrapper allows to assign OK, WARNING, CRITICAL and UNKNOWN - status to different strings. All in all, this check is designed to fulfil all monitoring tasks on modern web servers to meet the growing demand of monitoring web based services.


Requirements

This check is part of a plug in which has to be installed on the ARANSEC or CustoSec system.

CustoSec / ARANSEC must be allowed to contact the web server. In case of external web servers firewalls and routers have to be configured accordingly.

caption

The web site, that should be monitored, has to be configured as a host in ARANSEC / CustoSec. On this host, this check is configured as a service check.

Note: It is possible to treat a website like a host by entering the URL instead of the IP-address in the host definition.


Arguments

To configure the check, the following arguments are available:

Argument No. Argument Name Allowed Arguments Explanation Examples
Arg1 host fqdn | addr. | - Host name argument for servers using host headers (virtual host). Append a port to include it in the header (eg: example.com:5000). Fully qualified domain name or IP-Address of the Host that is to be monitored.
No empty field allowed.
Try first with no other parameters set. Check should deliver an "OK" state when the host is set correctly.
192.168.77.17
localhost
www.myserver.com
www.custosec.org
www.tagesschau.de
Arg2 port integer | - Web / Proxy Server port number (Default: 80). Any integer is allowed or "-" if not used.
No empty field allowed.
-
Arg3 ssl y | integer | s | - Connect to Server via SSL with default port 443. Use "y" to connect via ssh with auto-negotiation or use 1/2/3 to connect with "1"=TSLv1, "2"=SSLv2 or "3"=SSLv3 to connect without auto-negotiation.
Append "s" to enable SSL/TLS Hostname extension support ("s"=use SNI).
No empty field allowed.
ys
2s
y
- (no SSL)
Arg4 auth string:string:string[,..] | - Authentication for sites and proxies with basic authentication (i.e. Login is managed by a htaccess page).
Can be "web:username:password" and/or "proxy:username:password" or "-" if no authentication is needed.
No empty field allowed.
web:user:pass
web:user:pass,proxy:user:pass
- (for no authentication)
Arg5 method string | - Set the HTTP-Method. This can be all http-methods like CONNECT, HEAD, OPTIONS, TRACE, PUT, DELETE, etc.(Note: Use the http CONNECT - Method to tunnel ssl through through a proxy server).
No empty field allowed.
Arg6 tags string[,...] | - Any tags to be sent in http header. Use a comma separated list.
No empty field allowed.
User-Agent: Mozilla,Content-Type: text/html,charset=UTF-8,...
Arg7 nobody y | - Don't wait for document body: stop reading after headers. This makes the check a little bit faster, but it does not reduce the amount of data transferred since the check still does an HTTP GET or POST (not a HEAD).
No empty field allowed.
Arg8 timeout integer | - Seconds before connection times out (default: 10).
No empty field allowed.
User-Agent: Mozilla,Content-Type: text/html,charset=UTF-8,...
Arg9 path string | - URL to GET or POST (default: /). NOTE: some servers do need the "http://hostname-"part which has been given in Arg1 already. Depending on the configuration of the web server this needs some trial and errors.
No empty field allowed; use "-" for default.
Arg10 post string | - URL encoded http POST data.
Use this argument to enter the path to static websites.
Use this argument as well for logins on older login scripts (i.e. login.cgi) that need to receive the username and password URL encoded.
No empty field allowed.
username=Albert&password=Camus#11?13
Arg11 redirect string | - How to handle redirected pages (ok, warning, critical, follow, sticky, stickyport). Sticky is like follow but stick to the specified IP address. Stickyport also ensures port stays the same.
Arg12 age seconds | - Returns CRITICAL if document is more than SECONDS old. The number can also be of the form "10m" for minutes, "10h" for hours, or "10d" for days.
It depends on how the website is configured. If the website does not deliver a modification date in the header, the check will return "Document modification date unknown" and turn into state "CRITICAL".
10d
Arg13 size integer[:..] | - Minimum page size required (bytes) : Maximum page size required (bytes). Returns WARNING if page size is too small or too large. 264
Arg14 response double[,..] | - Response time to result in warning or critical status (seconds). 0.0005,0.0008
Arg15 search string | - Search arguments (See below) ss:HTTP
hs:WARN:Age:&OK:&

Search Arguments

The search arguments can address both, strings and regular expressions, in the header of a html-response or in the content. There is a simple search available which checks for strings or multiple strings in the header or the content of the web server response. Besides that there is a "status-wrapper", which will assign different strings to the 4 states (OK, WARN(ing), CRIT(ical) or UNKN(own).


The simple search

Syntax: <search-type:expression> or <-> for no search at all.

Search Type Syntax Explanation
String Search < ss: > Searches a string or a comma-separated list of strings (given as "expression") in the Header or the Content of the servers response.
At least one of the strings is expected in the first (status) line of the server response (default: HTTP/1.). If this search type is specified, it skips all other status-line logic (in the status wrapper, like processing the 3xx, 4xx, 5xx responses).
Header Search < hs: > "expression" must be a string to expect in the response headers (message header, HTTP-header). And only there!
Note: The http-header is not the <head></head>-Section of the html-site. Depending on the web servers configuration it contains technical information for the client browser. Learn more about HTTP-Protocol. To find out, which information is returned in the header of a website, a lot of online tools are available, like this HTTP Header Query
Content Search <cs:> "expression" must be a string to expect in the message body (content)
RegEx Search <rs[n|c|i]:> ". At least one match is expected in the page content (HTTP-header or message body) for the check to return an "OK". This allows to monitor a page after any kind of information. Find more information on regular expressions and the standard POSIX-Syntax.
The "n" attribute: Allows the regular expression to span new lines.
The "c" attribute: Defines the search as case insensitive.
The "i" attribute: Returns CRITICAL as Check result instead of OK in case the expression has been found.
Note: A combination of the optional regex switches is possible (eg: rsnci:).
Note: "i" has no effect if status:expression& wrapper is used (see blow).


The status wrapper

Syntax: <search-type:state:expression&[state:expression&,...]> or < - > for no search at all.
The status wrapper is working with "status:expression&"-pairs.

State Syntax Explanation
OK < OK: > Searches a string (given as "expression") in the Header or the Content of the servers response.
If the string is found, the check will return an OK state.
WARNING < WARN: > Searches a string (given as "expression") in the Header or the Content of the servers response.
If the string is found, the check will return an WARNING state.
CRITICAL < CRITL: > Searches a string (given as "expression") in the Header or the Content of the servers response.
If the string is found, the check will return an CRITICAL state.
UNKNOWN < UNKN: > If no corresponding expression matches, UNKNOWN is returned.
This behaviour can be changed by making the last "status:expression&"-pair

to match always (eg:rs:...&OK:.*& or cs:...&OK:&)

Note: For every "status:expression&"-pair, a HTTP request is sent to the web server in the order of the arguments. The first search expression match triggers the result. All remaining http requests are skipped.


Examples

The following examples should explain the usage of the check and how to enter the arguments
Please note: For layout reasons, the arguments line is displayed in multi lines in the "example" column.

Example/Description Output
!wikis.feosc.org!-!-!-!-!-!-!-!/index.php?title=CustoSec:Check_SNMP!-!-!-!-!-!-
This is a simple example to check if the page "CustoSec:Check_SNMP" exists on this wiki.
Please note: All other arguments that are not used on this query must be deactivated by using the "-" (minus) character.
If the page is found, the output of the check will be "OK" and a bunch of response times and the size.<br<In case the page is not found the check will return a "Warning" together with times and size of the web servers "404 page not found" - page
Status: OK
Output: HTTP OK: HTTP/1.1 302 Found - 264 bytes in 0.002 second response time
time=0.002396s;;;0.000000
size=264B;;;0
time_connect=0.000059s;;;
time_headers=0.000026s;;;
time_firstbyte=0.002277s;;;
time_transfer=0.002283s;;;
!test.feosc.org!-!-!web:aranea:#23ak+512fq?%mkt#!-!-!-!-!http://test.feosc.org/scripts/formmail.php?testalert=1!-!ok!-!-!-!cs:Test message sent.
Checking of a php-script named "formmail.php" on "www.test.feosc.org/scripts/". To access the directory "scripts" that is protected by a .htaccess, a user (aranea) and password (#...#) have to be entered. The configuration of the script, when called with "testalert=1, is to answer with a string "Text message sent. Check your email." (and send an email with some information).
The check is configured on a host "Website_Testing" with the webaddress "www.test.feosc.org" instead of the IP-address in the host configuration.

Please note: FormMail is a free and very advanced contact form processor provided by Tectite.com. Read more on Tectite's Website.
Status: OK
Output: HTTP OK: HTTP/1.1 200 OK - 427 bytes in 0.812 second response time
time=0.812381s;;;0.000000 size=427B;;;0
time_connect=0.276655s;;;
time_headers=0.000007s;;; time_firstbyte=0.459577s;;;
time_transfer=0.535696s;;;
!www.tagesschau.de!-!-!-!-!-!-!-!-!/ausland/usa-republikaner-101.html!-!-!300!0.0005,0.0008!-
Checking for size and response time of a page on the German News Channel site "Tagesschau.de".
Looking at the result, we see the check has found the page, but this too small, which would lead to a WARNING state.
Since the response time is about 0.053 seconds, which is far above the CRITICAL threshold of 0.008 seconds the check returns a CRITICAL state.
Status: CRITICAL
Output: HTTP CRITICAL: HTTP/1.1 302 Found - page size 231 too small - 231 bytes in 0.054 second response time |time=0.053724s;0.000500;0.000800;0.000000 size=231B;300;0;0 time_connect=0.000041s;;; time_headers=0.000024s;;; time_firstbyte=0.053606s;;; time_transfer=0.053617s;;;
!www.tagesschau.de!-!-!-!-!-!-!-!/index.php!-!-!-!-!-!hs:Content-Type: text/html
Checking if the string "Content Type: text/html" is returned in the header of a page on the German News Channel site "Tagesschau.de". Status: OK
Output: HTTP OK: HTTP/1.1 200 OK - 153276 bytes in 0.179 second response time
...
!www.tagesschau.de!-!-!-!-!-!-!-!/ausland/super-tuesday-101.html!-!-!-!-!-!hs:Content-Type: text/html
Same as above, but this time a deeper link (to a news page) is checked. Status: OK
Output: HTTP OK: HTTP/1.1 200 OK - 69105 bytes in 0.111 second response time
...
!www.tagesschau.de!-!-!-!-!-!-!-!/ausland/super-tuesday-101.html!-!-!-!-!-!cs:Bei den Republikanern werden heute rund ein Viertel aller
On the same page as above (a German news page about the US elections /Super Tuesday) the check tries to find a string (almost a complete sentence). Status: OK
Output: HTTP OK: HTTP/1.1 200 OK - 69105 bytes in 0.217 second response time
...
!192.168.34.122!-!-!-!-!-!y!-!/login.cgi!username=''username''&password=''password#78''!-!-!-!-!hs:Set-Cookie: WebManaged=2A0EF28C2E
This time the check is used to monitor a switch within the internal network. The question was, has a cookie (with a specific name) been set, after login to the web interface.
In this case Username and Password are "POST"ed within the URL, which means, we can use argument 10 to login into the web interface
Note: Special characters in the password are allowed excluding the reserved characters for the monitoring system (!,|,/,...). They are coded correctly by the check, so they can be entered without any masks.
In this example Argument 7 is set to y to stop the check reading after the header information. Section"
Status: OK
Output: HTTP OK: HTTP/1.0 302 Found - 456 bytes in 0.044 second response time
...


Useful Hints

  • Monitoring websites can be quite tricky, since it depends very much on the web server and his set up, how information is managed. This means, it sometimes will take some trial and error, to get the results. A good way to start is to begin with only the server address (Argument 1) and then the deep link (Argument 9) to check if a connection can me made. Only then other arguments should be built into the check.
  • The http-header information that this check is reading is not the information in the website's <head></head> tags! The http-header is information sent by the web server to tell receiving browsers what to expect and what to do. Normally this information cannot be seen.
    To help configure the check, it is a good idea, to find a tool, showing this information. There are many available on the internet. For Firefox browsers "Live HTTP headers" is a recommended extension.
  • If the Check returns only a "WARNING" or "CRITICAL - No Data received from host, this always means there is a problem with connecting to the server and finding the right URL.


Popular Pitfalls

This check is very flexible and can monitor almost everything on a web server. The following list collects some popular problems with working on this check.

  • When accessing web servers with a basic authentication, the password on the web server must not include the following characters: "!" (exclamation mark),|" (Pipe character), ";" (Colon). They are essential for NAGIOS to read and understand the sequence of commands.
  • When checking a website based on a Wiki software (like this website based on MediaWiki) a simple "Pages exists" check like in the first example will not work. Reason is that the wiki will automatically create the page (that is not existing yet) and ask if it should be edited.


Known Limitations

The check cannot deal with session cookies. This makes is difficult to monitor dynamic websites and logins. A new version of this check will be part of the next update (Early 2016).