Installing and Using SPLUNK on Ubuntu to index/monitor audit, snort, router and any other logs on my Linux system(s)

Recently, I started using SPLUNK for most of my log monitoring activities and I wanted to document some of my actions for future reference.

The first Step is to Install SPLUK on Ubuntu.

You will have to register with the website to obtain this download files and it will come with a temporary enterprise license, and after this license expires it will default to the 500 Mb per day license, without some of the enterprise functionality. This free license is still useful for monitoring logs on sytems that do not require more than 500Mb per day of data processing
.
http://www.splunk.com/download/?ac=ga0508_s_splunk&_kk=download%20splunk&_kt=0d0ee68f-00b5-441f-966d-e7b6c9228ad1&gclid=CLT9oK6jnLECFQ5Thwod-HNhfw

I installed and utilized the following version on my 32 bit system :
splunk-4.3.3-128297-linux-2.6-intel.deb

Once SPLUNK is installed you will need to start the service with the following commands  
sudo /opt/splunk/bin/splunk start

You can also stop and restart with the following commands:
sudo /opt/splunk/bin/splunk start¦ restart| stop

Go to the following location to access the web interface:

http://localhost:8000/

The web interface will provide a logon screen with the following default login credentials:

Login = admin

Password = changeme

Once you enter the above information, you will be given the change to change the default password

Once this all done then you will be take to the following screen -

Click on App -> *.nix

This app will provide most capabilities necessary to manage your host system - 

• Search
• OverviewDropdown
• CPUDropdown
• MemoryDropdown
• DiskDropdown
• NetworkDropdown
• UsersDropdown
• Log FilesDropdown
• ConfigsDropdown
• Setup

Help| About

I have been utilizing the Log Files functionality to manage my audit logs, snort logs, and pretty much all of my logs in "/var/log/*"

As  you can see below - rkhunter, audit.log and I am also monitoring my routers logs - 

• 
  
  Search
• OverviewDropdown
• CPUDropdown
• MemoryDropdown
• DiskDropdown
• NetworkDropdown
• UsersDropdown
• Log FilesDropdown
• ConfigsDropdown
• Setup

Help| About

Log Files Overview

|ActionsDropdown

index=os

DropdownAll time

Log Sources

• « prev 
• 1
 
• 2
 
• 3
 
• 4
 
• 5
 
• 6
 
• 7
 
• 8
 
• 9
 
• 10
 
• next »

Sort by source | totalCount (desc)

/var/log/dpkg.log.5.gz 48129

/var/log/dpkg.log.1 27803

/var/log/kern.log.1 12661

/var/log/rkhunter.log.4.gz 12283

/var/log/rkhunter.log.3.gz 12283

/var/log/rkhunter.log.2.gz 11275

/var/log/audit/audit.log 11129

/var/log/kern.log.0 10938

/var/log/dpkg.log 9087

/var/log/messages.0 8154

/var/log/dist-upgrade/term.log 8066

/var/log/dist-upgrade/apt-term.log 7934

/var/log/rkhunter.log 7626

/var/log/dist-upgrade/20120428-1918/term.log 6950

/var/log/dist-upgrade/20120428-1918/apt-term.log 6820

/var/log/upstart/ureadahead.log.1.gz 6614

/var/log/dist-upgrade/20111229-0955/term.log 6383

/var/log/dist-upgrade/20111229-0955/apt-term.log 6260

/var/log/kern.log.2.gz 5924

/var/log/kern.log.3.gz 5457

The best part of splunk's capabilities is the application does not care what type of data it indexes, you can pretty much index any type of data.

The dashboard and command interface are huge benefits that SPLUNK provides. 

You can enter any commands on the bar and any object you select in the web interface it will automatically build a command string for future use or reference, 

• 
  Search
• OverviewDropdown
• CPUDropdown
• MemoryDropdown
• DiskDropdown
• NetworkDropdown
• UsersDropdown
• Log FilesDropdown
• ConfigsDropdown
• Setup

Help| About

Log Files Overview

|ActionsDropdown

index=os Matching searches index=os /var/log/snort/alert alertindex=os /var/log/snort/alertindex=os source="/var/log/snort/alert"index=os source="/var/log/advanced.log"index=os source="/var/log/audit/audit.log" Matching terms index="os" How to Search Step 1: Retrieve EventsThe simplest searches return events that match terms you type into the search bar: terms: error login quoted phrases: "database error" boolean operators: login NOT (error OR fail) wildcards: fail* field values: status=404, status!=404, or status>200 Step 2: Use Search CommandsMore advanced searches use commands to transform, filter, and report on the events you retrieved. Use the vertical bar"|" , or pipe character, to apply a command to the retrieved events.

I've barely scratched the surface of SPLUNK's capabilities and usefulness and I will post in future blogs my Snort Dashboard and some great reports that it provides through the pre-built *nix and advanced searches- 

I have spent many years importing the data in MySQL and writing programs, C, C++, PERL, Python, etc... parsing these logs and importing the results in EXCEL for these types of reports, and really appreciate these pre-built capabilities.

Useful Links -
Splunk Log Analysis: Overview of cheat sheets, documents and other useful resources (Google Docs)
https://docs.google.com/document/d/1RXKG0NsnGApEu4mBlQV_frGEWb8V95UMYKkyOU7oJK0/edit?pli=1

Using Python and Shred to securely remove directories and files on a Linux File System - Updated

Creating an android app on an Ubuntu Platform with android SDK and Google's Python Android scripting

Creating an Android app using Ubuntu and PYTHON

Platform OS:

Ubuntu 10.10

Programming Language:

Python - SLA4 Android Scripting

Java JDK

Summary:

The latest mobile application trend motivated me to create an Android app utilizing python, and this will be a great way to increase my python programing skills. I do not have an application in mind yet but as soon as my platform is completed and I am running code on the emulator. I will start thinking about an application to develop.

Steps:

1. Download the openjdk-6-jdk package for Ubuntu 10.10
   http://packages.ubuntu.com/maverick/i386/openjdk-6-jdk/download
2. Install the package using  sudo dpkg -i openjdk-6-jdk_6b20-1.9.5-0ubuntu1_i386.deb or you can just use the file browser and click on the package and it will automatically install these packages

3. Determine if you have this package installed javac -version

1. javac -version and java -version 

4. Download the Android SDK “android-sdk_r09-linux_x86.tgz” from the following website  http://developer.android.com/sdk/index.html
5. Once you have this SDK install use the following command to untar the package on your Ubuntu platform.

1. tar zxvf android-sk_r09-linux_x86.tgz

6. Move in the un-compressed directory

1. cd android-sdk-linux_x86/tools

7. Execute the following command in the Android SDK folder

1. ./android

8. Click on the installed packages options and install all available packages

9. Then click on the Available packages and select both repositories and install selected. This will provide a wider array of platforms and expecially the Google add on.

10. Once you have the necessary add-ons just select a platform that you want to develop on and in my case I have chosen - 

11. Now for the programming portion of this build -  I will only create a simple androd hello world but will work on my official android app.
12. The first step is to install python capabilities on the android emulator

1. In the android browser go to the following website and download SL4A
2. code.google.com/p/android-scripting

13. Double click on the QR box and it will automatically download on the android simulator and then double click or tap on the SL4A download and you will be presented with the following action - press or double click on the INSTALL button

14. Once you have the SL4A - locate the installed application and double click on the interface and  you will have the scripting environment.
15. The next step is to install the python_for_android_r1.apk - Just double click on the link and it start the download and once the apk is downloaded - click on the .apk link and install .apk

16. After step 15 is performed look for the application on your menu area and select the Install button and the python_r7.zip file will download and extract on your android emulator. These are all of the support file for SL4A and some starting scripts.

17. Click on the test.py script and select command prompt screen

18. Run the hello world program and execute the following code :

        

19. Let get a GUI application running and then I can start thinking about what application I would provide the most benefits.
20. Created a script with the following code -

import android

droid = android.Android()

droidMsg = “My Test App”

droid.dialogCreateAlert(droidMsg)

droid.dialogSetPositiveButtonTexe(‘OK’)

droid.dialogShow()

resp = droid.dialogGetResponse().result

droid.makeToast(“Later”)

Conclusion

I am ready to starting coding my next application on the Android platform thanks to Google for releasing these features and thanks to my favorite Linux Journal Magazine.