Collective Intelligence Framework v3 - Part 2

By Kunal Patil, Senior Threat Hunter


In Previous blog we learned how to setup CIFv3, in this section we will discuss how to integrate data and filter the Resulted Data by-passing different parameters to it.

We might have question that what happens exactly after hitting cif command or how cifv3 fetches feeds from external resources. well in this blog i will cover every details of CIFv3 but before that lets study few terminologies to understands the implementation of this Project.

CIFv3 uses few Terminologies like :

1) TLP

TLP stands for Traffic Light Protocol originally created by UK government for the Purpose of sharing of sensitive data.
There are four colors in TLP (same like traffic lights) :

  • RED - Not for disclosure, restricted to participants only
  • AMBER - Limited disclosure, restricted within own organisation and clients or customers.
  • GREEN -  Limited disclosure, restricted to the community but not via public channels
  • WHITE - Disclosure is not limited

2) Timestamps

CIF supports three Timestamps per record. each record is always associated with Timestamps like reporttime, lasttime, firsttime.

  • firsttime - The very first time record was observed
  • lasttime - when the Last time it was observed
  • reporttime - Time when record was given to you

3) Indicator

CIF uses Indicator types for filteration, as shown in below image data is filtered by URL. list of indicators are :

  • cif --itype ipv4 >> ipv4 address
  • cif --itype ipv6 >> ipv6 address
  • cif --itype fqdn >> fully qualified domain address
  • cif --itype url >> url address
  • cif --itype email >> email address
  • cif --itype md5 >> md5 hash
  • cif --itype sha1 >>sha1 hash
  • cif --itype sha256 >> sha256 hash
  • cif --itype sha512 >> sha512 hash
  • cif --itype uuid >> uuid hash



4) Country code

Query can process by by country code. every country is associated with the cc, for example US stands for United States, JP stands for JAPAN etc.



5) ASN

AS stands for Autonomous system. AS is a collection of group of networks which is controlled by common network administrator. ASN is Autonomous system number which is an identifier for the AS to uniquely identified globally.

Query or filter by ASN. A few examples:

  • cif --asn 36351
  • cif --asn 199789



6) Confidence

Confidence details are the degree of assurance/reliability of a given observation. by judging confidence may be subjective in case of observable's. cif uses 0-10 scale for assigning confidence. whitelists are used to help further reduce the risk of blocking something like google.com.

  • 0-4 - informational Data or machine generated data
  • 5 - 50/50 shot same like coin flip
  • 6 - Not confident eg:- you trust the data autor with root access
  • 7 - Somehow confident
  • 8 - very confident
  • 9 - 10 confirmed

command : cif --confidence 7
by hitting this command, it will display malwares which have confidence 7 or more than 7. 
command : cif --confidence 8
by hitting this command, it will display malwares which have confidence 8 or more than 8 and so on...



7) Tags

Tags are nothing but the labels given to the treat. Tags are not pre-defined in cif, instead we can create new tag by inserting into feed configuration file. Default tags shipped with CIF:

  • Botnet
  • exploit
  • hijacked
  • malware
  • phishing
  • scanner
  • search
  • suspicious
  • whitelist

eg : cif --tags malware



8) Related Data

Its mainly consists of IP address, if IP address is not available it will shows none

9) Provider

Provider refers to the source from where the all data has been fetch

  • cif --provider openphish.com



10) Group

Query of filter by group

11) Format

The CIF client can supports several different output formats:

  • cif -f table
  • cif -f json
  • cif -f csv
  • cif -f snort
  • cif -f bro
  • cif -f bind
  • cif -f html



12) Limit

CIF can Limit the number of results returned

  • cif -f table --limit 5



13) Search using FQDN, IP ,URL's and hashes

  • Search querry using command : cif -q


14) Whitelisting

CIF has the capability to whitelist observations from entering a feed during the feed generation process. By default CIF is configured with the following whitelists which helps for whitelisting the records.

a) alexa.yml
b) mirc.yml

Hence, We learned what all parameters are required to Understands Threat Data. Feeds configuration files are Preconfigured which comes with default CIF installation. CIF will load all feed configuration files found in /etc/cif/rules/default with the file extension .yml. Any files without the extension of .yml are ignored.



After hitting CIF command, CIF will redirects to path /etc/cif/rules/default and executes predefined Feeds configuration files. CIF runs all Scripts once and Displays gathered results in one file. we can also imports the feeds data to one file by using ">>" filename.extension.

Preconfigured Feeds data Script contains the Following Data :

figure: Data within Scripts

 In the Next blog, i will discuss how to create Feeds configuration files.


STAY TUNED .....



I am a security researcher where my expertize includes domains like incident response and investigation, Malware Analysis, Threat Hunting, deception Technology, Brand protection, SIEM configuration, Digital Forensics, Cyber Threat Intelligence, EDR, Network Security, Content Filtering, etc While working I had completed my Masters(2021) in Information security where I published a thesis on “Malware Analysis using machine Learning Ensemble” My goal is to share the knowledge with everyone with technologies, upcoming trends, various platforms and mainly Lock it down, protect it up, and block the hackers.

Comments

Post a Comment

Popular posts from this blog

RTR using Falcon Crowdstrike

By Kunal Patil, Senior Threat Hunter
Image
Real Time Response Real Time Response is a powerful tool that gives security administrations the ability to remotely access systems for administration tasks, remediation actions or forensics collection, etc. without requiring physical access to the system. For more information on the CrowdStrike solution, see the additional resources and links below. In the Falcon UI, navigate to Activity > Detections. Commonly, a new detection will be the event that triggers a need for remediation.Directly from a given detection, the “Connect to Host” button allows you to remotely connect and take action. You can also connect to a host from Hosts > Host Management. Once connected, you will be presented with a list of commands and capabilities available in Real Time Response. With the ability to run commands, executables and scripts, the possibilities are endless. A few examples are listed below.
Read more

Top Commands Mostly Used By System Administrator.

By Kunal Patil, Senior Threat Hunter
Image
Top Commands Mostly Used By System Administrator. 1) IPconfig IPCONFIG Simply entering ipconfig at the command line will return basic addressing information for your system, including the adapter name, IP address, subnet mask, and default gateway. 2) IPconfig /all IPCONFIG/ALL Shows all networking information for the system, including host name, node type, adapter names, MAC addresses, DHCP lease information, etc. 3) Ping PING Ping command to verify that a host can be reached over the network. This command is useful for diagnosing host and network connectivity problems. The device sends a series of ICMP echo (ping) requests to a specified host and receives ICMP echo responses. 4) Ping "Website-name" PING TO WEBSITE   This above command used to find the IP address associated with The Specific Website. eg: 10.24.126.25 is the IP address for Website Intranet.indusind.com 5) Tracert "IP address" TRACE COMMAND Tracert comm
Read more

SECURITY OPERATION CENTRE

By Kunal Patil, Senior Threat Hunter
Image
What is SOC?                    Security operations centers monitor and analyze activity on networks, servers, endpoints, databases, applications, websites, and other systems, looking for anomalous activity that could be indicative of a security incident or compromise. The SOC is responsible for ensuring that potential security incidents are correctly identified, analyzed, defended, investigated, and reported. SOC works with collaboration with People, Process and Technology.  SOC Architecture PEOPLE:                  The best way to think of a SOC is as a centralized team of people who provide threat monitoring, investigation, and response.  Larger SOCs employ a three-level analyst structure for handling security alerts generated by a security system or SIEM.  Level 1 analysts are responsible for real-time monitoring of security alerts, doing triage on them, and deciding whether an alert is serious enough to be escalated to a Level 2 analyst.  Level 1 analysts se
Read more

Damn Vulnerable Web Application - Part 1

By Kunal Patil, Senior Threat Hunter
Image
The Damn Vulnerable Web Application (DVWA) provides a PHP/MySQL web application that is damn vulnerable whose goal of being an intentionally vulnerable system for practice/teaching purposes in regard to Information Security.There are many Methods for Installing DVWA on Platforms Like Windows and Linux, In this blog i will show the easiest Walk-through for Beginners to Learn and Exploit Web Application. So let us Start with Installation and Implementation of DVWA. Step By Step Installation 1) Requirement :  Windows system for Managing VM Virtual Box ( Link ) or VMware ( Link ) should be installed on system Kali-ISO ( Link ) DVWA ISO ( Link ) 2) Switch-ON Kali and DVWA virtual machines Both must be in HOST only Adapter (we can change using VM Network setting).  3) After the Installation of Kali and DVWA in VM, find the IP address of DWVA using the Command ifconfig and Check Connectivity between them using ping command. 4) Run Kali machine and DVWA machine Par
Read more

Cyber Threat Intelligence

By Kunal Patil, Senior Threat Hunter
Image
Gartner Definition:   Threat intelligence is evidence-based knowledge, including context, mechanisms, indicators, implications and actionable advice, about an existing or emerging menace or hazard to assets that can be used to inform decisions regarding the subject's response to that menace or hazard.  To understand this term easily lets split the words and bring out meaning to it :  Cyber :  relating to or characteristic of the culture of computers, information technology, and virtual reality. Threat :  thing likely to cause damage or danger. Intelligence :  the ability to acquire and apply knowledge and skills.                 Cyber Threat intelligence helps Organisation to understands the risk of most common External threats. T hreat intelligence includes in-depth information about specific threats to help an organization protect itself from the types of attacks that could do them the most damage.  One of the proven methods to stay on top of attacks is to detect and
Read more

Collective Intelligence Framework v3 - Part 1

By Kunal Patil, Senior Threat Hunter
Image
Basic About CTI Cyber Threat intelligence(CTI) is a Technology which helps an Organization to collect and Analyze threat data received from multiple resources. Cyber threat intelligence is an automation process where it accumulate data from various external resources (such as FEEDS) and recognize the threats suitable for the Organization. By importing the Data from CTI, the next step is to exporting the CTI data into Existing Security systems. Collective intelligence Framework is an underlying Structure of CTI which helps any organization to gather all Threat Data at one place. In this blog we are discussing how to Install Collective Intelligence Framework v3 (Bearded Avenger) into security structure. Details Information About CIF you will found at : csirtgadgets Basic Requirements for Bearded Avenger CIF v3 : OS: Ubuntu 16 LTS,  x64 RAM: 16GB Cores: 4 (As Sqlite, ElasticSearch, CIF-Router among other apps would be running on same instance) HDD Capacity: 100GB
Read more

Top 20 Subdomains Search Engines

By Kunal Patil, Senior Threat Hunter
Image
Whats is Domain?  An organization shares a common suffix as Domain names which is controlled by that Organization or individual. What is Subdomains? Basically Subdomains are subdivisions of Domain. Why it is required? Subdomains are second website, which have its own unique content and makes hosting manageable and easy to organize.         Every second Organization Consists of various Subdomains as per their Requirements, I listed few Subdomains Search Engines, list is below :  Google Search - Online Search Censys - Online Search Pentest-Tools - Online Search DNS Dumpster - Online Search Netcraft - Online Search CloudPiercer - Online Search Detectify - Online Search VirusTotal - Online Search Pkey - Online Search Crt.sh - Online Search Sublist3r - GitHub repository DNScan - GitHub Repository Knock - GitHub Repository SubBrute - GitHub Repository Nmap - GitHub Repository Gobuster - GitHub Repository Fierce - Bydefault in Kali DNS
Read more

Security Architecture for Startup

By Kunal Patil, Senior Threat Hunter
Image
           Security Architecture is the design artifacts that describes how the security controls are positioned and how they relate to the overall systems architecture. These controls serve the Purpose to maintain the system's quality attributes such as confidentiality, integrity and availability. Following is the 10 steps plan required to build Security Architecture For Startup :   Pick Your battle  Establish a security culture  Pick security platform  Upgrade your software  Physical security  Control the internal network  Secure coding  Protect devices against malware  Perform security audits  BYOD Policies Pick Your Battle You can't secure everything, Quantify the monetory damage, likelyhood and mitigation cost of each threat to prioritize time and resource. Below are the list of threats which can taken into consideration as per Goal of the Industry (Budget + Risks). Establish a Security Culture Show Your team that Security is Important Factor
Read more

TrickBot Malware Family - A Deep Dive using Falcon Crowdstrike

By Kunal Patil, Senior Threat Hunter
Image
Use Case  Falcon OverWatch has identified probable TrickBot malware on host NAOUXXXX. A renamed copy of Vhd2disk was executed on the host, likely from a successful phish. This in turn wrote and executed a malicious file, likely TrickBot malware, and established persistence for it with a registry key. Introduction  Emotet Often referred to as a banking trojan or worm. It is a very advanced threat that is updated multiple times a day by the cybercrooks controlling it. Emotet is an advanced, modular banking Trojan that primarily functions as a downloader or dropper of other banking Trojans. Additionally, Emotet is a polymorphic banking Trojan that can evade typical signature-based detection. It has several methods for maintaining persistence, including auto-start registry keys and services. It uses modular Dynamic Link Libraries (DLLs) to continuously evolve and update its capabilities. TrickBot TrickBot is most commonly installed by the Emotet Trojan, which is spread through phishing em
Read more