Portfolio

Strategic Telecom Infrastructure Mapping Case Study for Altman Solon

Industry: Market Intelligence & Infrastructure Research
Altman Solon and their strategic research teams requiring granular, ground-truth infrastructure data to model market penetration and competitive displacement.

Problem

Strategic consulting for the telecom sector often hits the "Coverage Visibility Gap"—the inability to verify service availability at a massive scale beyond manual spot-checks.
  • Extreme Fragmentation: Fiber-optic coverage data is siloed across 10+ unique provider portals, each with its own UI logic, address-validation flow, and "anti-bot" defenses.
  • Scaling Bottleneck: The requirement to map 181,000+ postcodes against 10 distinct ISPs translates to nearly 2 million unique data points. Manual collection is impossible, and basic scripts fail at this volume.
  • Security & Rate-Limiting: Major telecom websites employ sophisticated Web Application Firewalls (WAFs) and rate-limiting to prevent high-volume automated lookups.
  • Temporal Decay: Infrastructure changes monthly. A one-time scrape provides a "dead" snapshot; the client needed a repeatable pipeline to track the "velocity of rollout."

Solution

A high-performance Infrastructure Intelligence Pipeline designed for high-concurrency data extraction and status normalization.
  • Multi-Architectural Extraction: A hybrid engine that combines direct API interception (where available) with headless browser automation to handle complex Javascript-heavy search forms.
  • Priority Queueing: A tiered execution model that allows for the "Fast-Track" delivery of high-priority Tier-1 providers while processing the broader provider list in parallel.
  • Intelligent Normalization: A logic layer that interprets diverse provider responses (e.g., "Coming soon," "Planned," "Serviceable but survey required") and converts them into a standardized Serviceability Matrix.
  • Adaptive Human-Emulation: Integration of rotating residential proxies and session management to maintain a high success rate without triggering automated blocks.

Tech Stack

  • Automation: Headless Browser Orchestration (Playwright / Selenium)
  • Backend: Python (Asynchronous Data Pipeline)
  • Intelligence: Custom NLP for address-match verification and status classification
  • Infrastructure: Distributed Task Queues & Residential Proxy Networks
  • Delivery: Normalized JSON/CSV optimized for GIS and BI tool integration

Results

  • High-Velocity Turnaround: Processed the entire metropolitan dataset (1.8M+ data points) with a delivery window of less than 7 days for the core priority providers.
  • Data-Driven Strategy: Enabled Altman Solon to provide their clients with 99%+ accurate infrastructure penetration maps, replacing estimates with verified ground-truth data.
  • Repeatable Partnership: The modularity of the code established a long-term collaboration model, allowing for quarterly "refreshes" of the dataset at a fraction of the initial development cost.
  • Operational Intelligence: Standardized disparate ISP data into a unified format, eliminating weeks of manual data cleaning for the client's internal analytical teams.
Web scraping & Market intelligence