OpenStack Private Cloud for AI & Microservices
Built a private cloud infrastructure using OpenStack as an open-source alternative to VMware ESXi. The system runs virtualized microservices and AI workloads across three VMs: Data Lake with MongoDB, AI Worker Node with TensorFlow and YARA, and Analytics with ELK Stack. Processed 267+ security events with 71.91% classified as Critical Threats using AI-driven threat detection achieving 0.941 beaconing detection score.
Technologies Used
Project Overview
This project focuses on building an OpenStack-based Private Cloud as an open-source alternative to proprietary hypervisors like VMware ESXi. The infrastructure is designed to run microservices and AI workloads simultaneously in an isolated, scalable, and easily managed environment, specifically for cybersecurity research and academic computing needs. By leveraging OpenStack as an IaaS platform, this project demonstrates that a self-managed cloud can provide high flexibility, cost efficiency, and full control over resources without dependency on commercial vendors.
System Architecture
VM 1 - Data Lake & Management: Central raw data storage running MongoDB for storing honeypot logs, attack data, and malware samples at scale using flexible NoSQL schema.
VM 2 - AI Worker Node: Primary compute node running Python, TensorFlow, Jupyter Notebook, and YARA for data preprocessing, static malware analysis, and machine learning-based threat classification.
VM 3 - Analytics & Visualization: Running ELK Stack (Elasticsearch & Kibana) for indexing analysis results and real-time security data visualization.
Key Features
Full Open-Source Private Cloud using OpenStack (Nova, Neutron, Keystone, Glance, Horizon) eliminating vendor lock-in and providing full infrastructure control.
Distributed AI Workload Processing with separation between data storage, AI processing, and visualization improving performance and scaling ease.
AI-Driven Threat Detection capable of detecting automatic beaconing patterns with detection score up to 0.941, distinguishing human and bot traffic.
Centralized Security Dashboard through Kibana for monitoring threat levels, attack trends, and malware activity distribution.
Scalable IaaS Architecture enabling rapid VM provisioning to support growing research needs.
System Flow
Data Collection: Honeypot (Cowrie) captures network attack activity and generates attack logs and malware data.
Data Ingestion & Storage: Raw data is centrally stored in MongoDB on VM Data Lake ensuring consistency and easy access.
Pre-processing Data: AI Worker Node retrieves data from MongoDB and performs normalization, metadata extraction, and data preparation for further analysis.
Malware Analysis: Static analysis using YARA for malware signature detection and Machine learning analysis using TensorFlow for identifying beaconing patterns and automated activity.
Threat Classification: Activities are classified into Low, Medium, and High/Critical Risk based on analysis scores.
Indexing & Visualization: Analysis results are sent to Elasticsearch and visualized through Kibana dashboard for monitoring and security insight extraction.
Project Outcome
✓ 267+ security events processed, 71.91% Critical Threats identified