…and stupid pet tricks.

I really don’t get the cost gap in NV scanning products that we are seeing.

We started using Tenable’s Nessus Pro product for doing our network vulnerability scanning. Mainly because our internal audit-appointed-pentesters were using Nessus, and this was making us look fairly stupid.  So our first step is…reproduce what the auditors are seeing.

This was all well and good, but then…I found it was hard to track our progress on fixing things.  And sometimes, we would fix things, and then have regressions because “oh, that box got reinstalled, and we forgot to change that setting…”  (yes, yes, config management tools, etc…we will get there.  some.  day).

So…our security guy started looking for “enterprise” products.  We looked at two.  I won’t name them…but…the jump in costs went from R30K (US$2300) / year to…10-20x that amount…I recall figures of R350K (US$25K) and R1M (US$75K) per year.

What did these products do…um…they did “continuous scanning” (e.g., they ran your scans on a schedule?).  And they had dashboards.  And…they promised baseline-vs-current reporting, but, I didn’t actually see this…and heard from a customer ref that this was coming “real soon now”.

Nessus REST API

But…our basic “non-enterprise” scanning solution has…a comprehensive and well-documented REST API

https://cloud.tenable.com/api#/overview

with some examples using CURL listed here

https://community.tenable.com/docs/DOC-1172

This was relatively easy to use with python + the well-known requests library.  I wrote a small class that handled creating / checking authentication tokens and the various API calls that I needed for launching scans, listing scans and scan histories and exporting and downloading scan data.

The first thing was to launch scans from a script that could then be scheduled with cron.  The hardest part of this was actually just getting the cron config to run…very tedious editing crontab, setting a time 5 mins from now, and waiting to see what happens.

And…once I got that working, I discovered that Nessus Pro actually allows you to associate a daily / weekly / monthly schedule with each scan definition…so the work with python + REST + cron was not really needed (but was educational).

After that, wrote a “downloadNewScans” script in python that:

• queries a DB table for the UUIDs of all previously downloaded scans
• gets a list of all scan definitions
• for each scan, gets a history of all previous runs of the scan
• checks if the run is a new (unseen) UUID
• if so, then requests Nessus to export the scan as a Nessus XML file
• polls the status of the export until ready
• downloads the file and saves it under a filename that identifies the run unabiguously with the scanid and historyid of the run.

Scan Results – XML

The scan data can be exported in Nessus XML format, which is documented here

https://static.tenable.com/documentation/nessus_v2_file_format.pdf

But even without these docs, the XML in the files is easily readable.

I used python + xml.minidom to parse the XML files — which do get fairly large, a single class-C network in our data centre range is about 8MB of XML data, and the campus-wide scans, that cover 30-40 subnets are about 90MB of XML data.

I wrote the results into a simple schema in a MySQL database that allows me to start asking questions and producing the reports that I want from SQL queries.  Here is one of my first stabs at tracking critical vulnerabilities in a subnet on a weekly basis: