Web Scraping MLS – Camilo Sierra's Portfolio

WebScraping MLS‑2024

Data exploration and analysis of the 2024 Major League Soccer season. The 2024 Major League Soccer (MLS) season provided an excellent opportunity to explore team performance, player statistics, and match outcomes. This project aimed to collect, clean, and analyze data to uncover actionable insights about the league.

Methodology

Para llevar a cabo este análisis, se implementaron las siguientes etapas:

Data Collection & Preparation: Gathered official MLS data using Python tools such as requests (HTTP requests), BeautifulSoup (HTML parsing), and glob (batch file handling), loading everything into pandas DataFrames for further processing.

Data Cleaning: Addressed missing values, standardized data formats, and validated datasets to ensure accuracy.

Exploratory Analysis: Used statistical summaries and visualizations to identify key patterns.

Visualization: Created interactive dashboards to communicate findings effectively.

import pandas as pd import numpy as np import glob import seaborn as sns import plotly.express as px import matplotlib.pyplot as plt plt.style.use('default') import warnings from tabulate import tabulate warnings.filterwarnings('ignore')

1. Data Collection

To gather the necessary data for this project, we utilized web‑scraping techniques. The pandas library in Python provides the read_html function, which was employed to extract structured data directly from the MLS website. The specific URL targeted contains the scores and fixtures for the 2024 Major League Soccer season. By specifying the id attribute of the desired table (sched_all), we ensured precise extraction of the relevant dataset. This approach allows us to work with a clean and organized table for further analysis.

Wk	Day	Date	Time	Home	xG	Score	xG (Away)	Away	Attendance	Venue	Match Report
1	Wed	2024-02-21	20:00	Inter Miami	1.4	2–0	0.8	Real Salt Lake	21 137	Chase Stadium	Match Report
1	Sat	2024-02-24	13:45	LAFC	1.5	2–1	1.8	Seattle Sounders	22 214	BMO Stadium	Match Report
1	Sat	2024-02-24	14:00	Columbus Crew	1.8	1–0	0.5	Atlanta Utd	20 406	Lower.com Field	Match Report
1	Sun	2024-02-25	14:30	FC Cincinnati	0.9	0–0	0.5	Toronto FC	25 513	TQL Stadium	Match Report
1	Sun	2024-02-25	16:00	Nashville SC	0.1	0–0	1.4	NY Red Bulls	30 109	Geodis Park	Match Report
1	Sat	2024-02-24	19:30	Austin	1.1	1–2	3.0	Minnesota Utd	20 738	Q2 Stadium	Match Report
1	Sat	2024-02-24	19:30	Orlando City	0.9	0–0	1.0	CF Montréal	24 249	Inter&Co Stadium	Match Report
1	Sat	2024-02-24	19:30	Portland Timbers	0.5	4–1	1.2	Colorado Rapids	21 411	Providence Park	Match Report
1	Sat	2024-02-24	19:30	Houston Dynamo	1.0	1–1	0.2	Sporting KC	18 818	Shell Energy Stadium	Match Report
1	Sat	2024-02-24	19:30	Philadelphia Union	2.1	2–2	1.6	Chicago Fire	18 734	Subaru Park	Match Report

Day

Date

Time

Home

Score

xG (Away)

Away

Attendance

Venue

Match Report

Wed

2024-02-21

20:00

Inter Miami

1.4

2–0

0.8

Real Salt Lake

21 137

Chase Stadium

Match Report

Sat

2024-02-24

13:45

LAFC

1.5

2–1

1.8

Seattle Sounders

22 214

BMO Stadium

Match Report

Sat

2024-02-24

14:00

Columbus Crew

1.8

1–0

0.5

Atlanta Utd

20 406

Lower.com Field

Match Report

Sun

2024-02-25

14:30

FC Cincinnati

0.9

0–0

0.5

Toronto FC

25 513

TQL Stadium

Match Report

Sun

2024-02-25

16:00

Nashville SC

0.1

0–0

1.4

NY Red Bulls

30 109

Geodis Park

Match Report

Sat

2024-02-24

19:30

Austin

1.1

1–2

3.0

Minnesota Utd

20 738

Q2 Stadium

Match Report

Sat

2024-02-24

19:30

Orlando City

0.9

0–0

1.0

CF Montréal

24 249

Inter&Co Stadium

Match Report

Sat

2024-02-24

19:30

Portland Timbers

0.5

4–1

1.2

Colorado Rapids

21 411

Providence Park

Match Report

Sat

2024-02-24

19:30

Houston Dynamo

1.0

1–1

0.2

Sporting KC

18 818

Shell Energy Stadium

Match Report

Sat

2024-02-24

19:30

Philadelphia Union

2.1

2–2

1.6

Chicago Fire

18 734

Subaru Park

Match Report

2. Data Cleaning

This step ensures that all values in the Date column are converted into a standard datetime format, allowing for consistent handling of dates across the dataset. By using pd.to_datetime() with errors='coerce', invalid or improperly formatted dates are replaced with NaT (Not a Time), preventing errors during the conversion process. This transformation is crucial for enabling chronological analysis, filtering by date, and performing time‑based calculations efficiently.

Index	Round	Wk	Day	Date	Time	Home	xG	Score	xG (Away)	Away	Attendance	Venue
15	— all NaN —
30	— all NaN —
44	— all NaN —
59	— all NaN —
73	— all NaN —
88	— all NaN —
103	— all NaN —
118	— all NaN —
133	— all NaN —
148	— all NaN —

To ensure the dataset contained only valid match rows, we applied two focused filters to the Date column. First, MLS = MLS[MLS["Date"].notna()] removes every row where the parsed value is NaT—these are blank separator lines that FBref inserts between rounds. Second, MLS = MLS[MLS["Date"] != "Date"] drops the occasional row where the literal text "Date" reappears as a repeated header inside the table. After these two steps, the DataFrame contains only genuine match records with proper datetime values, eliminating noise and preventing downstream errors in time‑based analyses.

3. Exploratory Analysis: Messi economic impact on MLS

The table below lists a sample of MLS matches with the core fields we scraped and cleaned: week/day/date/time, home and away teams, each side’s xG, the final score, attendance, and venue. We also derived Home Goals, Away Goals, Total Match Goals, and a Goal Bucket label. This gives quick context for each game and shows the raw inputs we’ll use to build the KPI in the next section.

Wk	Day	Date	Time	Home	xG	Score	xG (Away)	Away	Attendance	Venue	Match Report	Home Goals	Away Goals	Total Match Goals	Goal Bucket	0 Goals	1 Goal	2 Goals	3 Goals
1	Wed	2024-02-21	20:00	Inter Miami	1.4	2–0	0.8	Real Salt Lake	21 137	Chase Stadium	Match Report	2	0	2	2	0	0	1	0
1	Sat	2024-02-24	13:45	LAFC	1.5	2–1	1.8	Seattle Sounders	22 214	BMO Stadium	Match Report	2	1	3	3	0	0	0	1
1	Sat	2024-02-24	14:00	Columbus Crew	1.8	1–0	0.5	Atlanta Utd	20 406	Lower.com Field	Match Report	1	0	1	1	0	1	0	0
1	Sun	2024-02-25	14:30	FC Cincinnati	0.9	0–0	0.5	Toronto FC	25 513	TQL Stadium	Match Report	0	0	0	0	1	0	0	0
1	Sun	2024-02-25	16:00	Nashville SC	0.1	0–0	1.4	NY Red Bulls	30 109	Geodis Park	Match Report	0	0	0	0	1	0	0	0

To compute this KPI, we classify each match by total goals, clean bad records, and aggregate by outcome to see how scoring intensity distributes across results.

Bucket: Turn the raw Score into a Goal Bucket (0, 1, 2, 3, or 4+) for every match.
Clean: Filter out rows where parsing failed (wrong dash, missing, or non-numeric values).
Group: Aggregate by Outcome (Home / Draw / Away), summing each goal bucket and counting Total Matches.
Read: Quickly see scoring distribution by result and convert to percentages to support pricing, marketing, and scheduling decisions.

KPI — Goals by Outcome
Outcome	0 Goals	1 Goal	2 Goals	3 Goals	4+ Goals	Total Matches
Away	0	29	16	54	59	158
Draw	30	0	46	0	44	120
Home	0	33	40	68	94	235
Total	30	62	102	122	197	513

We filtered all matches where Inter Miami played (home or away). For each game, we calculated the team’s goals scored, then grouped by Outcome (Home / Draw / Away) and by Goal Bucket (0, 1, 2, 3, 4+). The bar chart below counts how many club matches fall into each combination, so you can quickly see whether wins cluster in 2–3 or 4+ goal games, whether draws tend to be low-scoring, and how the pattern shifts home vs. away. This view helps quantify a possible “Messi effect”: more high-scoring matches and more favorable results.

Illustrative image of web scraping process for MLS data

Camilo Sierra E.

WebScraping MLS‑2024

Methodology

We import the necessary libraries:

1. Data Collection

2. Data Cleaning

3. Exploratory Analysis: Messi economic impact on MLS

WebScraping MLS‑2024

Methodology

We import the necessary libraries:

1. Data Collection

2. Data Cleaning

3. Exploratory Analysis: Messi economic impact on MLS

WebScraping MLS‑2024