What you should Have to Start

Lesson Portion 1: ReIntroduction to Data Analysis, NunPy, and Pandas, Why is it important?

Data Analysis.

  • Data Analysis is the process of examining data sets in order to find trends and draw conclusions about the given information. Data analysis is important because it helps businesses optimize their performances.

What is NunPy and Pandas

  • Pandas library involves a lot of data analysis in Python. NumPy Library is mostly used for working with numerical values and it makes it easy to apply with mathematical functions.
  • Imagine you have a lot of toys, but they are all mixed up in a big box. NumPy helps you to put all the same types of toys together, like all the cars in one pile and all the dolls in another. Pandas is like a helper that helps you to remember where each toy is located. So, if you want to find a specific toy, like a red car, you can ask Pandas to find it for you.
  • Just like how it's easier to find a toy when they are sorted and organized, it's easier for grown-ups to understand and analyze big sets of numbers when they use NumPy and Pandas.

Lesson Portion 2 More into NunPy

What we are covering;

  • Explanation of NumPy and its uses in data analysis
  • Importing NumPy library
  • Examining NumPy arrays
  • Creating NumPy arrays and performing intermediate array operations
  • Popcorn Hacks, Make your own percentile NunPy array

What is NunPy's use in data analysis/ how to import NunPy.

NumPy is a tool in Python that helps with doing math and data analysis. It's great for working with large amounts of data, like numbers in a spreadsheet. NumPy is really good at doing calculations quickly and accurately, like finding averages, doing algebra, and making graphs. It's used a lot by scientists and people who work with data because it makes their work easier and faster.

import numpy as np

List of NunPy Functions, what they do, and examples.

Example of Using NunPy in Our Project

This code calculates the total plate appearances for a baseball player using NumPy's sum() function, similar to the original example. It then uses NumPy to calculate the total number of bases (hits plus walks) for the player, and divides that by the total number of plate appearances to get the on-base percentage. The results are then printed to the console.

import numpy as np

# Example data
player_hits = np.array([3, 1, 2, 0, 1, 2, 1, 2])  # Player's hits in each game
player_walks = np.array([1, 0, 0, 1, 2, 1, 1, 0])  # Player's walks in each game
player_strikeouts = np.array([2, 1, 0, 2, 1, 1, 0, 1])  # Player's strikeouts in each game

# array to store plate appearances (PA) for the player
total_pa = np.sum(player_hits != 0) + np.sum(player_walks) + np.sum(player_strikeouts)

# array to store on-base percentage (OBP) for the player
total_bases = np.sum(player_hits) + np.sum(player_walks)
obp = total_bases / total_pa

# Print the total plate appearances and on-base percentage for the player
print(f"Total plate appearances: {total_pa}")
print(f"On-base percentage: {obp:.3f}")
Total plate appearances: 21
On-base percentage: 0.857

Activity 1; PopCorn Hacks; Creating a NunPy Array and Analyzing the Data using Array Operations

import numpy as np

#Create a NumPy array of the heights of players in a basketball team
heights = np.array([192, 195, 193, 200, 211, 199, 201, 198, 184, 190, 196, 203, 208, 182, 207])

# Calculate the percentile rank of each player's height
percentiles = np.percentile(heights, [25, 50, 75])

# Print the results
print("The 25th percentile height is", percentiles[0], "cm.")
print("The 50th percentile height is", percentiles[1], "cm.")
print("The 75th percentile height is", percentiles[2], "cm.")

# Determine the number of players who are in the top 10% tallest
top_10_percent = np.percentile(heights, 90)
tallest_players = heights[heights >= top_10_percent]

print("There are", len(tallest_players), "players in the top 10% tallest.")
The 25th percentile height is 192.5 cm.
The 50th percentile height is 198.0 cm.
The 75th percentile height is 202.0 cm.
There are 2 players in the top 10% tallest.
import numpy as np

#Create a NumPy array of the heights of players in a basketball team
Numbers = np.array([202, 300, 190, 250, 141, 195, 201, 198, 184, 990, 236, 253, 207, 112, 207])

# Calculate the percentile rank of each player's height
percentiles = np.percentile(Numbers, [25, 50, 75])

# Print the results
print("The 25th percentile Numbers is", percentiles[0], "Percent.")
print("The 50th percentile Numbers is", percentiles[1], "Percent.")
print("The 75th percentile Numbers is", percentiles[2], "Percent")

# Determine the number of players who are in the top 10% tallest
top_10_percent = np.percentile(Numbers, 90)
Highest_number = Numbers[Numbers >= top_10_percent]

print("There are", len(Highest_number), "Numbers in the top 10% Highest.")
The 25th percentile Numbers is 192.5 Percent.
The 50th percentile Numbers is 202.0 Percent.
The 75th percentile Numbers is 243.0 Percent
There are 2 Numbers in the top 10% Highest.

Lesson Portion 3 More into Pandas

What we are Covering

  • Explanation of Pandas and its uses in data analysis
  • Importing Pandas library
  • Loading data into Pandas DataFrames from CSV files
  • Manipulating and exploring data in Pandas DataFrames
  • Example of using Pandas for data analysis tasks such as filtering and sorting

What are pandas and what is its purpose?

  • Pandas is a software library that is used in Python
  • Pandas are used for data analysis and data manipulation
  • Pandas offer data structures and operations for manipulating numerical tables and time series.
  • Pandas is free

Things you can do using pandas

  • Data Cleansing; Identifying and correcting errors, inconsistencies, and inaccuracies in datasets.
  • Data fill; Filling in missing values in datasets.
  • Statistical Analysis; Analyzing datasets using statistical techniques to draw conclusions and make predictions.
  • Data Visualization; Representing datasets visually using graphs, charts, and other visual aids.
  • Data inspection; Examining datasets to identify potential issues or patterns, such as missing data, outliers, or trends.

Pandas and Data analysis

The 2 most important data structures in Pandas are:

  • Series ; A Series is a one-dimensional labeled array that can hold data of any type (integer, float, string, etc.). It is similar to a column in a spreadsheet or a SQL table. Each element in a Series has a label, known as an index. A Series can be created from a list, a NumPy array, a dictionary, or another Pandas Series.
  • DataFrame ;A DataFrame is a two-dimensional labeled data structure that can hold data of different types (integer, float, string, etc.). It is similar to a spreadsheet or a SQL table. Each column in a DataFrame is a Series, and each row is indexed by a label, known as an index. A DataFrame can be created from a dictionary of Series or NumPy arrays, a list of dictionaries, or other Pandas DataFrame.

Dataframes

import pandas as pd
pd.__version__
'1.4.2'

Importing CSV Data

  • imports the Pandas library and assigns it an alias 'pd'.
  • Loads a CSV file named 'nba_player_statistics.csv' into a Pandas DataFrame called 'df'.
  • Specifies a player's name 'Jimmy Butler' to filter the DataFrame for that player's stats. It creates a new DataFrame called 'player_stats' which only contains rows where the 'NAME' column matches 'Jimmy Butler'.
  • Displays the player's stats for points per game (PPG), rebounds per game (RPG), and assists per game (APG) using the print() function and string formatting.
  • The code uses the double square brackets [[PPG', 'RPG', 'APG']] to select only the columns corresponding to the player's points per game, rebounds per game, and assists per game from the player_stats DataFrame.
  • In summary, the code loads NBA player statistics data from a CSV file, filters it for a specific player, and displays the stats for that player's PPG, RPG, and APG using a Pandas DataFrame.
import pandas as pd
# Load the CSV file into a Pandas DataFrame
df = pd.read_csv('/Users/taiyoi/Documents/Compsci22-1/_notebooks/nba_player_statistics.csv')
# Filter the DataFrame to only include stats for a specific player (in this case, Jimmy Butler)
player_name = 'Jimmy Butler'
player_stats = df[df['NAME'] == player_name]
# Display the stats for the player
print(f"\nStats for {player_name}:")
print(player_stats[['PPG', 'RPG', 'APG']])
Stats for Jimmy Butler:
    PPG  RPG   APG
0  35.0  5.0  11.0

In this code segment below we use Pandas to read a CSV file containing NBA player statistics and store it in a DataFrame.

The reason Pandas is useful in this scenario is because it provides various functionalities to filter, sort, and manipulate the NBA data efficiently. In this code, the DataFrame is filtered to only include the stats for the player you guys choose.

  • Imports the Pandas library and assigns it an alias 'pd'.
  • Loads a CSV file named 'nba_player_statistics.csv' into a Pandas DataFrame called 'df'.
  • Asks the user to input a player name using the input() function and assigns it to the variable player_name.
  • Filters the DataFrame for the specified player name using the df[df['NAME'] == player_name] syntax, and assigns the resulting DataFrame to the variable player_stats.
  • Checks if the player_stats DataFrame is empty using the empty attribute. If it is empty, prints "No stats found for that player." Otherwise, it proceeds to step 6.
  • Displays the player's stats for points per game (PPG), rebounds per game (RPG), assists per game (APG), and total points + rebounds + assists (P+R+A) using the print() function and string formatting.
  • In summary, this code loads NBA player statistics data from a CSV file, asks the user to input a player name, filters the DataFrame for that player's stats, and displays the player's stats for PPG, RPG, APG, and P+R+A. If the player is not found in the DataFrame, it prints a message indicating that no stats were found.
import pandas as pd
df = pd.read_csv('/Users/taiyoi/Documents/Compsci22-1/_notebooks/nba_player_statistics.csv')
# Load CSV file into a Pandas DataFrame
player_name = input("Enter player name: ")
# Get player name input from user
player_stats = df[df['NAME'] == player_name]
# Filter the DataFrame to only include stats for the specified player
if player_stats.empty:
    print("No stats found for that player.")
else:
# Check if the player exists in the DataFrame
    print(f"\nStats for {player_name}:")
print(player_stats[['PPG', 'RPG', 'APG', 'P+R+A']])
# Display the stats for the player inputted.
No stats found for that player.
Empty DataFrame
Columns: [PPG, RPG, APG, P+R+A]
Index: []

Lesson Portion 4

What we will be covering

  • Example of analyzing data using both NumPy and Pandas libraries
  • Importing data into NumPy and Pandas Performing basic data analysis tasks such as mean, median, and standard deviation Visualization of data using Matplotlib library

Example of analyzing data using both NumPy and Pandas libraries

import numpy as np
import pandas as pd

# Load CSV file into a Pandas DataFrame

df = pd.read_csv('/Users/taiyoi/Documents/Compsci22-1/_notebooks/nba_player_statistics.csv')

# Filter the DataFrame to only include stats for the specified player

player_name = input("Enter player name: ")
player_stats = df[df['NAME'] == player_name]
if player_stats.empty:
    print("No stats found for that player.")
else:

    # Convert the player stats to a NumPy array
    player_stats_np = np.array(player_stats[['PPG', 'RPG', 'APG', 'P+R+A']])

    # Calculate the average of each statistic for the player

    player_stats_avg = np.mean(player_stats_np, axis=0)

    # Print out the average statistics for the player

    print(f"\nAverage stats for {player_name}:")
    print(f"PPG: {player_stats_avg[0]:.2f}")
    print(f"RPG: {player_stats_avg[1]:.2f}")
    print(f"APG: {player_stats_avg[2]:.2f}")
    print(f"P+R+A: {player_stats_avg[3]:.2f}")
No stats found for that player.

NumPy impacts the given code because it performs operations on arrays efficiently. Specifically, it converts a Pandas DataFrame object to a NumPy array object, and then calculates the average statistics for a the player you guys inputted. Without NumPy, it would be more difficult and less efficient to perform these calculations on large data sets. It does the math for us.

Importing data into NumPy and Pandas Performing basic data analysis tasks such as mean, median, and standard deviation Visualization of data using Matplotlib library

Matplotlib is used essentially to create visuals of data. charts,diagrams,etc.

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt

# Load the CSV file into a Pandas DataFrame
df = pd.read_csv('/Users/taiyoi/Documents/Compsci22-1/_notebooks/nba_player_statistics.csv')

# Print the first 5 rows of the DataFrame
print(df.head())

# Calculate the mean, median, and standard deviation of the 'Points' column
mean_minutes = df['MPG'].mean()
median_minutes = df['MPG'].median()
stddev_minutes = df['MPG'].std()

# Print the results
print('Mean Minutes: ', mean_minutes)
print('Median Minutes: ', median_minutes)
print('Standard Deviation Minutes: ', stddev_minutes)

# Create a histogram of the 'Points' column using Matplotlib
plt.hist(df['MPG'], bins=20)
plt.title('MPG Histogram')
plt.xlabel('MPG')
plt.ylabel('Frequency')
plt.show()
   RANK             NAME TEAM POS   AGE  GP   MPG  USG%   TO%  FTA  ...   APG  \
0     1     Jimmy Butler  Mia   F  33.6   1  42.9  34.3   9.9    8  ...  11.0   
1     2    Kawhi Leonard  Lac   F  31.8   2  40.2  30.0  11.9   17  ...   6.0   
2     3  Khris Middleton  Mil   F  31.7   1  33.1  37.5  19.8   10  ...   4.0   
3     4     Devin Booker  Pho   G  26.5   2  44.1  28.8  16.2   14  ...   6.0   
4     5     De'Aaron Fox  Sac   G  25.3   2  38.2  31.6   9.0   14  ...   7.0   

   SPG  BPG  TPG   P+R   P+A  P+R+A    VI   ORtg   DRtg  
0  3.0  0.0  3.0  40.0  46.0   51.0  11.6  117.2  103.8  
1  2.0  0.5  3.0  41.0  40.5   47.0  11.0  129.5  110.4  
2  0.0  0.0  5.0  42.0  37.0   46.0  12.8  115.5  111.9  
3  2.5  1.5  4.0  33.0  38.0   39.0   5.2  121.9  111.0  
4  3.5  0.5  2.5  34.0  38.0   41.0   9.1  112.6  108.8  

[5 rows x 29 columns]
Mean Minutes:  20.985483870967748
Median Minutes:  23.0
Standard Deviation Minutes:  12.844102823170283

In this example code, we first import the necessary libraries, including NumPy, Pandas, and Matplotlib. We then load the CSV file into a Pandas DataFrame using the pd.read_csv() function. We print the first 5 rows of the DataFrame using the df.head() function. Next, we calculate the mean, median, and standard deviation of the 'MPG' column using the appropriate Pandas methods, and print the results. And, we create a histogram of the 'MPG' column using Matplotlib by calling the plt.hist() function and setting appropriate axis labels and a title. We then call the plt.show() method to display the plot. Even though NumPy is not directly used in this code, it is an important underlying component of the pandas and Matplotlib libraries, which are used to load, manipulate and visualize data. It allows them to work more efficiently

Lesson Portion 5; Summary

Summary/Goals of Lesson:

One of our goals was to make you understand data analysis and how it can be important in optimizing business performance. We also wanted to make sure you understood the use of Pandas and NumPy libraries in data analysis, with a focus on NumPy. As someone who works with data, we find Pandas incredibly useful for manipulating, analyzing, and visualizing data in Python. The way we use pandas is to calculate individual player and team statistics. We are a group that works with numerical data, so NumPy is one of our favorite tools for working with arrays and applying mathematical functions to them. It is very fast at computing and manipulating arrays making it a very valuable tool for tracking statistics which is important to our group. For example, if you have an array of the points scored by each player in a game, you can use NumPy to calculate the total points scored, average points per player, or the highest and lowest scoring players.

Lesson Portion 6 Hacks

Printing a CSV File (0.5)

  • Use this link https://github.com/ali-ce/datasets to select csv file of a topic you are interested in, or you may find one online.
  • Once you select your topic make sure it is a csv file and then you want to press on the button that says raw.
  • After that copy that information and create a file with a name and .csv at the end and paste your information.
  • Below is a start that you can use for your hacks.
  • Your goal is to print 2 specific parts from data (example could be like population and country).

Popcorn Hacks (0.2)

  • Lesson Portion 1. #### Answering Questions (0.2)
  • Found Below.

Submit By Thursday 8:35 A.M.

  • How to Submit: Slack a Blog Post that includes all of your hacks to "Joshua Williams" on Slack.
import numpy as np
import pandas as pd

# Load CSV file into a Pandas DataFrame

df = pd.read_csv('/Users/taiyoi/Documents/Compsci22-1/_notebooks/Starbucks World Stats.csv')

# Display the first five rows of the dataframe
print(df.head())

# Get summary statistics for numerical columns
print(df.describe())

# Filter the dataframe to show only rows with Country/Region equals "United States"
us_df = df[df['Country/Region'] == 'United States']
print(us_df.head())

# Group the dataframe by Continent and show the mean values for each column
continent_df = df.groupby('Continent').mean()
print(continent_df.head())
          Country  Population  Numer of Starbucks  \
0   United States   316128839               11851   
1           China  1357380000                1462   
2          Canada    35158304                1425   
3           Japan   127338621                1052   
4  United Kingdom    64097085                 770   

   Starbucks per million inhabitants  
0                              37.49  
1                               1.08  
2                              40.53  
3                               8.26  
4                              12.01  
         Population  Numer of Starbucks  Starbucks per million inhabitants
count  6.300000e+01           63.000000                          63.000000
mean   8.035355e+07          327.634921                           6.144444
std    2.316215e+08         1505.995709                           8.997930
min    3.783100e+04            1.000000                           0.040000
25%    5.419304e+06            9.000000                           0.865000
50%    1.546820e+07           26.000000                           2.160000
75%    5.715838e+07          103.500000                           7.595000
max    1.357380e+09        11851.000000                          40.530000
---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
File ~/opt/anaconda3/lib/python3.9/site-packages/pandas/core/indexes/base.py:3621, in Index.get_loc(self, key, method, tolerance)
   3620 try:
-> 3621     return self._engine.get_loc(casted_key)
   3622 except KeyError as err:

File ~/opt/anaconda3/lib/python3.9/site-packages/pandas/_libs/index.pyx:136, in pandas._libs.index.IndexEngine.get_loc()

File ~/opt/anaconda3/lib/python3.9/site-packages/pandas/_libs/index.pyx:163, in pandas._libs.index.IndexEngine.get_loc()

File pandas/_libs/hashtable_class_helper.pxi:5198, in pandas._libs.hashtable.PyObjectHashTable.get_item()

File pandas/_libs/hashtable_class_helper.pxi:5206, in pandas._libs.hashtable.PyObjectHashTable.get_item()

KeyError: 'Country/Region'

The above exception was the direct cause of the following exception:

KeyError                                  Traceback (most recent call last)
/Users/taiyoi/Documents/Compsci22-1/_notebooks/2023-04-19-tri3_lesson.ipynb Cell 42 in <cell line: 15>()
     <a href='vscode-notebook-cell:/Users/taiyoi/Documents/Compsci22-1/_notebooks/2023-04-19-tri3_lesson.ipynb#X56sZmlsZQ%3D%3D?line=11'>12</a> print(df.describe())
     <a href='vscode-notebook-cell:/Users/taiyoi/Documents/Compsci22-1/_notebooks/2023-04-19-tri3_lesson.ipynb#X56sZmlsZQ%3D%3D?line=13'>14</a> # Filter the dataframe to show only rows with Country/Region equals "United States"
---> <a href='vscode-notebook-cell:/Users/taiyoi/Documents/Compsci22-1/_notebooks/2023-04-19-tri3_lesson.ipynb#X56sZmlsZQ%3D%3D?line=14'>15</a> us_df = df[df['Country/Region'] == 'United States']
     <a href='vscode-notebook-cell:/Users/taiyoi/Documents/Compsci22-1/_notebooks/2023-04-19-tri3_lesson.ipynb#X56sZmlsZQ%3D%3D?line=15'>16</a> print(us_df.head())
     <a href='vscode-notebook-cell:/Users/taiyoi/Documents/Compsci22-1/_notebooks/2023-04-19-tri3_lesson.ipynb#X56sZmlsZQ%3D%3D?line=17'>18</a> # Group the dataframe by Continent and show the mean values for each column

File ~/opt/anaconda3/lib/python3.9/site-packages/pandas/core/frame.py:3505, in DataFrame.__getitem__(self, key)
   3503 if self.columns.nlevels > 1:
   3504     return self._getitem_multilevel(key)
-> 3505 indexer = self.columns.get_loc(key)
   3506 if is_integer(indexer):
   3507     indexer = [indexer]

File ~/opt/anaconda3/lib/python3.9/site-packages/pandas/core/indexes/base.py:3623, in Index.get_loc(self, key, method, tolerance)
   3621     return self._engine.get_loc(casted_key)
   3622 except KeyError as err:
-> 3623     raise KeyError(key) from err
   3624 except TypeError:
   3625     # If we have a listlike key, _check_indexing_error will raise
   3626     #  InvalidIndexError. Otherwise we fall through and re-raise
   3627     #  the TypeError.
   3628     self._check_indexing_error(key)

KeyError: 'Country/Region'

Question Hacks;

NumPy: NumPy is a Python library for numerical computation. It provides a powerful array object that allows you to efficiently store and manipulate large arrays of homogeneous data. NumPy is widely used in data analysis for performing mathematical operations on arrays, such as linear algebra, Fourier transform, and random number generation.

Pandas: Pandas is a Python library built on top of NumPy that provides high-level data structures and functions designed for easy data manipulation and analysis. It provides a DataFrame object, which is similar to a spreadsheet or a SQL table, and allows you to easily perform data cleaning, merging, filtering, and aggregation.

Differences between NumPy and Pandas: While both NumPy and Pandas are used in data analysis, they have different use cases. NumPy is used primarily for numerical computation and scientific computing, while Pandas is used for data manipulation and analysis. NumPy provides a powerful array object that allows you to perform mathematical operations on large arrays of homogeneous data, while Pandas provides a DataFrame object that allows you to manipulate and analyze data in a more flexible and structured way.

DataFrame: A DataFrame is a 2-dimensional labeled data structure with columns of potentially different types. It is similar to a spreadsheet or a SQL table and provides a flexible and powerful tool for data manipulation and analysis. In a DataFrame, each column can have a different data type, such as integer, float, or string.

Common operations with NumPy: NumPy provides a wide range of functions for mathematical operations on arrays, such as arithmetic operations, linear algebra, Fourier transform, and random number generation. Some common operations include creating arrays, indexing and slicing arrays, performing mathematical operations, reshaping arrays, and computing statistical measures.

Incorporating NumPy or Pandas into a project: To use NumPy or Pandas in a project, you need to install them first using pip or conda package manager. Once installed, you can import the library into your project and start using its functions and data structures. For example, to create a NumPy array, you can use the np.array() function, and to create a Pandas DataFrame, you can use the pd.DataFrame() function. You can then use various functions and methods provided by the libraries to manipulate and analyze the data.