How an ISP Reaches the Global Internet: Upstreams, IXPs, and Peering

When an enterprise subscriber or residential user connects to an Internet Service Provider (ISP) and opens a web service hosted on another continent, packets must traverse dozens of independent networks. The global internet is not a single centrally managed entity, but a decentralized mesh of more than 100,000 Autonomous Systems (AS) bound together by economic agreements and standardized exterior gateway protocols [1].

In this technical guide, we examine how an ISP establishes global reachability, analyze the mechanics of Autonomous System Numbers (ASNs) and Border Gateway Protocol version 4 (BGP-4), and evaluate the distinctions between IP transit, Internet Exchange Points (IXPs), and private peering [1, 3].

1. Autonomous Systems and the Hierarchy of ISPs

An Autonomous System (AS) is a collection of connected IP routing prefixes under the control of one or more network operators that presents a common, clearly defined routing policy to the internet [1]. Every commercial ISP operates one or more globally registered Autonomous System Numbers (ASNs) assigned by regional internet registries such as APNIC or RIPE NCC.

The global internet topology is structured into three informal tiers:

• Tier 1 Carriers: Global backbone networks (such as Lumen, Telia/Arelion, NTT, and Tata Communications) that own transoceanic subsea cables and extensive terrestrial fiber plants. Tier 1 networks peer with all other Tier 1 providers without paying for transit (settlement-free peering). They do not purchase transit from any third party.
• Tier 2 Providers: Regional and national ISPs that own substantial physical fiber networks within a country or geographic region. Tier 2 ISPs peer locally with peers to exchange intra-regional traffic, but must purchase upstream IP transit from Tier 1 providers to reach the full global routing table.
• Tier 3 (Access) ISPs: Local access providers, broadband operators, and campus networks that deliver "last-mile" connectivity to end consumers and commercial subscribers. Tier 3 operators typically purchase all external reachability from upstream Tier 2 or Tier 1 providers.

2. Upstream IP Transit Mechanics

IP transit is a commercial, bilateral service agreement where an upstream provider commits to carrying traffic between the customer ISP and any routable address on the global internet.

BGP Session Establishment

To initiate transit, the customer ISP establishes an External BGP (EBGP) peering session over a dedicated physical circuit (such as a 10G or 100G optical cross-connect) with the upstream carrier's edge router [1].

1. TCP Port 179: The peering routers establish a stateful TCP connection on port 179.
2. OPEN Message Exchange: The routers negotiate parameters including BGP version (BGP-4), ASN identifiers, BGP Hold Timers, and Multi-Protocol extensions (IPv4 and IPv6 unicast).
3. Prefix Announcements: The customer ISP advertises its globally routable IP prefix blocks (for example, a public /22 IPv4 block or /32 IPv6 block) to the upstream carrier. The upstream carrier propagates these advertisements across its global network, announcing to the world that it can deliver packets to the customer's IP addresses [1].
4. Full Table vs Default Route: The upstream provider either sends a "Full Internet Routing Table" (exceeding 950,000 IPv4 routes and 200,000 IPv6 routes as of 2026) or a lightweight Default Route (0.0.0.0/0). Full routing tables allow the customer ISP to make optimized, latency-based forwarding decisions across multiple transit providers.

Transit Billing and the 95th Percentile

Unlike residential broadband, commercial IP transit is rarely billed on fixed data volume caps. Instead, enterprise transit is typically metered using 95th percentile burstable billing:

• The upstream provider samples interface utilization every 5 minutes throughout a 30-day billing month (yielding roughly 8,640 samples).
• At the end of the month, the top 5% highest peak traffic samples are discarded.
• The customer is billed based on the 95th percentile threshold. This model accommodates transient bandwidth spikes without penalizing the ISP with exorbitant burst charges.

3. Internet Exchange Points (IXPs) and Peering

While IP transit guarantees universal reachability, relying exclusively on paid upstreams is economically inefficient. If two competing ISPs in the same city exchange traffic solely through international upstreams, local packets may route overseas and back (hairpinning), creating unnecessary latency and high transit fees.

An Internet Exchange Point (IXP) solves this by providing a neutral physical layer-2 switching fabric where multiple local ISPs, content delivery networks (CDNs), and cloud providers interconnect to exchange traffic directly [2].

Public Peering at an IXP

At an IXP facility, participating networks connect their edge routers to a high-density, multi-terabit Ethernet switching platform:

• Bilateral Peering: Two network operators negotiate an agreement to exchange traffic directly across the IXP switch fabric.
• Multilateral Peering (Route Servers): Rather than establishing hundreds of individual BGP sessions with each member, an ISP connects to the IXP's redundant Route Servers. An announcement to the Route Server propagates to all other participating peers automatically, radically simplifying peering operations [1].

Private Network Interconnect (PNI)

When traffic volume between two specific networks becomes exceptionally heavy—such as an ISP streaming petabytes of video traffic from a major hyper-scaler—public IXP ports can saturate. The two entities establish a Private Network Interconnect (PNI): a dedicated physical optical link directly between their routers, bypassing the shared IXP switch fabric.

Comparing Transit and Peering

Preventing BGP Route Leaks and Hijacks

BGP operates fundamentally on mutual trust, making it vulnerable to accidental misconfigurations and malicious hijacking [2].

The Hazard of Route Leaks

A catastrophic scenario occurs when an ISP accidentally re-advertises routes learned from one upstream provider to another upstream provider or across a settlement-free peering exchange. The ISP effectively tells the world, "Send your global internet traffic through my small network!" This causes massive link saturation, packet dropping, and global outages [2].

Engineering Defenses

In accordance with modern internet standards, every ISP must implement strict routing hygiene:

1. RFC 8212 Strict Filtering: Ensure BGP implementations reject route propagation by default unless an explicit outbound route-map or prefix-list permits it [3].
2. RPKI (Resource Public Key Infrastructure): Cryptographically sign Route Origin Authorizations (ROAs) through regional registries. Border routers validate incoming BGP advertisements against RPKI crypto databases, dropping invalid hijack attempts before they pollute the FIB.
3. Strict Ingress Prefix Filtering: Apply explicit prefix-list and AS-Path regex filtering on all customer and peering sessions to prevent unauthorized announcements [2].

Designing and maintaining carrier-grade BGP infrastructure requires deep routing expertise and continuous monitoring. Explore our engineering services on our NOC & Network Operations Support page.

Key Takeaways

• An ISP achieves global connectivity through a balanced combination of paid IP transit and settlement-free peering.
• BGP-4 is the exterior routing protocol that exchanges reachability information between Autonomous Systems.
• Internet Exchange Points (IXPs) keep local traffic local, reducing transit operational expenditure and latency.
• Strict BGP route filtering (RFC 8212) and RPKI Route Origin Validation are mandatory to prevent catastrophic route leaks.

Need help with this? Contact us.

Sources

[1] IETF, "RFC 4271: A Border Gateway Protocol 4 (BGP-4)", 2006. URL: https://www.rfc-editor.org/rfc/rfc4271.html (The core standards document defining BGP finite state machines, path attributes, and exterior routing convergence).

[2] IETF, "RFC 7908: Problem Definition and Classification of BGP Route Leaks", 2016. URL: https://www.rfc-editor.org/rfc/rfc7908.html (Defines formal classifications of routing leaks and multi-homed ISP configuration vulnerabilities).

[3] IETF, "RFC 8212: Default External BGP (EBGP) Route Propagation Behavior Without Policies", 2017. URL: https://www.rfc-editor.org/rfc/rfc8212.html (Standards-track requirement mandating default-deny route propagation rules on EBGP sessions).

Last reviewed: October 2026