C-Rex Internals and Developer Notes

How to run for development locally

    git clone https://bitbucket.org/netnea/c-rex.git

    # backend
    cd express_app
    npm install # once after checkout
    npm start

    # frontend
    cd react_app
    npm install # once after checkout
    npm start

    # robotframework tests, served by react_app development node.js
    cd robotframework
    robot [ --variable BROWSER:(Chrome|Firefox)] --variable LOGIN_PORT:3000 c-rex.robot

    # if robotframework is not installed yet
    python -m venv robotframework/venv
    . robotframework/venv/bin/activate
    pip install robotframework robotframework-selenium2library webdrivermanager
    (optional, only if no driver present yet) webdrivermanager BROWSER 

    # example using venv and limiting run to a specific test in Firefox
    venv/bin/robot --variable LOGIN_PORT:3000 -t "Generate Rule runtime rule tag"  c-rex.robot

Now the react_app node instance can be accessed at http://localhost:3000.
It proxies ot th express_app backend instance on 3001

How to run/build/test in Docker

The Dockerfile creates a static build of the frontend, which will then be served by the backend node.js on port 3001 from /c-rex.

The port 3000 react node.js is not running in the docker image.


    docker build --no-cache --tag c-rex .  
    docker run --rm -it -p 3303:3001 c-rex:latest 

    # or, to not conflict with vagrant already running
    docker run --rm -it -p 4303:3001 c-rex:latest

    # to run the tests in the final image
    docker run --rm -it -p 3303:3001 --env mode=test c-rex:latest

    # run a subset of tests (testfilter is passed to the -t parameter in robotframework)
    # mount robotlog on host so files can be inspected later
    docker run --rm -it --env mode=test --env testfilter="*param" --mount type=bind,source=/tmp/robotlog,target=/tmp/robotlog c-rex:latest 

    # to poke around in the container filesystem
    docker run --rm -it -p 3303:3001 --entrypoint=bash c-rex:latest

Run from CI VM on Proxmox

The latest master, develop and feature/* branches are always published at:

This runs on the ramci Proxmox VM, and should always be the Docker build of the latest version. The proxy is on the next Proxmox VM, and requires basic auth. Instructions:

If you’d like to have an account, login via sysadmin@neajump1 to the VM next, and add an account to /etc/apache2/rpxypasswd

To access the Jenkins server on ramci:

https://rpxy.netnea.com:22443/jenkins/

Run in MiniShift

Set up Virtual Machine Manager with KVM support
Ensure your user is member of the relevant groups
Download latest MiniShift release from MiniShift’s Github project
Extract archive and change into directory
Start MiniShift: ./minishift start
Open console: ./minishift console
Add c-rex: Add to Project -> Deploy Image -> Image Name
Add route for deployment

modsec-rulereport.rb

Most of the work is done by express_app/modsec-rulereport.rb. This file should be never edited, but instead imported from the master repository https://git.netnea.com:10443/folini/modsec-rulereport.rb/src/master/bin/modsec-rulereport.rb

Local Storage Configuration Options

c-rex.loglevel = trace|debug|info|warn|error - set console.log verbosity, defaults to warn

Supported URL Parameters

?logline=<urlencoded data> - pass the log entry by url parameter instead of form input, in urlencoded format.
?logline_base64=<base64 encoded data> - pass the log entry by url parameter instead of form input, in base64 format. The encoded bytes must be plain ASCII.

ui states

await_input
error
exclusion (log parsed successfully)
advisory

Automatic Action Configuration (draft, not yet implemented in express_app)

The application-wide config file is TBD.

Configure if the generated rule can be manually edited in the UI before submitting to the action

config.x.rule_editable = true|false

The scripts/programs application can be configured in wemo/config/application.rb, example:

config.x.actions = [ 
    { :label => "Deploy to Dev", :action =>  "/mnt/project/action_example1.sh dev" },
    { :label => "Deploy to Int", :action =>  "/mnt/project/action_example1.sh int" },
    { :label => "Request Prod Approval", :action =>  "/mnt/project/action_example1.sh prod param1 param2" }
]

`action` Interface

Any custom program/script can be used as action. After any command line parameters defined in the configuration, an additional temporary filename like /tmp/modsec-rule20190301-16349-145mfoa will be passed. This is a JSON file containg the original logline (in base64), the parsed logline fields, the chosen action and finally the generated rule:

{
  "action": {
    "label": "Deploy to Int",
    "action": "/mnt/project/action_example1.sh int"
  },
  "rule": "      # ModSec Rule Exclusion: 942410 : SQL Injection Attack\\n      SecRuleRemoveById 942410\\n",
  "ruleid": 1015,
  "logline_base64": "WzIwMTYtMTEtMDMgMjI6NTQ..snip..l0=",
  "logline_fields": {
    "id": "942410",
    "unique_id": "WBuyJX8AAQEAAEdWTgQAAACL",
    "ip": "127.0.0.1",
    "msg": "SQL Injection Attack",
    "uri": "/drupal/index.php/search/node",
    "parameter": "ARGS:keys",
    "hostname": "localhost",
    "file": "/apache/conf/owasp-modsecurity-crs-3.0.0-rc1/rules/REQUEST-942-APPLICATION-ATTACK-SQLI.conf",
    "tags": [
      "OWASP_CRS/WEB_ATTACK/SQL_INJECTION",
      "WASCTC/WASC-19",
      "OWASP_TOP_10/A1",
      "OWASP_AppSensor/CIE1",
      "PCI/6.5.2"
    ]
  }
}

This allows to write flexible action scripts that can integrate into any given environment. Note that the JSON file only exists during the execution of the action, it is unlinked after the action terminates.

Common exit code behaviour applies:

0 will show in the UI as success, everything else as an error
on exit 0, any potential stdout will be displayed in the UI
on exit != 0, stderr will be displayed in the UI