Flappy-Ball RL

Flappy-Ball RL is a reinforcement learning demo where hundreds of neural-network agents learn a Flappy Bird-style game through evolutionary algorithms. It started as a curiosity about how simple networks can discover control policies, and grew into a playground to compare optimization strategies, visualize learning in real time, and benchmark design choices.

Purpose and Overview

The project showcases population-based learning with genetic algorithms and alternatives (ES, DE, PSO). Agents read a compact game state and decide when to jump. Fitness is based on survival and pipe passing, with shaping terms that encourage smooth flight and robustness.

Zoom image +Training viewport with live stats

Key Features

• Population Training: 50-500 agents evolve in parallel
• Real-time Visualization: See improvement across generations
• Interactive Controls: Tune physics, NN, and evolution at runtime
• Live Analytics: Fitness trends, generation stats, best agent playback
• Persistence: Save and load top genomes
• Deterministic Mode: Seeded RNG for reproducible runs

What's Inside the AI

Neural Network

• Inputs: position/velocity, next gap info, distance to ceiling/ground (6-8 features)
• Hidden layers: 8 → 4 with tanh
• Output: sigmoid → jump probability

Learning Algorithms

• Genetic Algorithm (default): tournament selection, uniform crossover (~70%), adaptive mutation, elite preservation, wider weight bounds [-5, 5]
• Evolution Strategy (μ+λ): self-adaptive mutation for fine-tuning
• Differential Evolution: diversity-aware exploration for continuous params
• Particle Swarm: quick convergence via velocity + global/personal bests

Zoom image +Algorithm and physics controls

Game Mechanics

• Physics: gravity + impulse jump; terminal velocity
• Pipes: constant gap size, smooth random vertical drift, fixed spacing
• Fitness: survival distance + gap-centering bonus + pipe-pass reward - movement penalty
• Determinism: procedural generation seeded for consistency

Zoom image +Live training metrics

Technology Stack

• React 19 + TypeScript (UI and state)
• Vite (build and dev)
• Canvas 2D API (rendering at 60 FPS)
• Vitest (tests)
• Zero external ML deps (custom NN, GA/ES/DE/PSO, RNG, math)

Architecture at a Glance

• game/nn.ts — forward pass and genome ops
• game/evolve.ts — genetic algorithm core
• game/world.ts — physics, collisions, procedural pipes
• game/sim.ts — training loop, generation orchestration
• game/render.ts — canvas rendering and culling

Getting Started

Prerequisites

• Node.js 18+
• pnpm (recommended) or npm

Install and Run

git clone [email protected]:ponytojas/Flappy-Ball-RL.git
cd flappy-ball-rl
pnpm install
pnpm dev      # open http://localhost:5173
pnpm test     # run unit tests
pnpm build    # production build

Live Demo

You can play around with it here.