DadOverflow.com

Musings of a dad with too much time on his hands and not enough to do. Wait. Reverse that.

Tag: python (Page 7 of 26)

Learning Guitar with Python, Part 3

May 3, 2021 /

Past posts in this sorta-series: Part 1 and Part 2.

Pentatonic scales are quite popular among guitar players. If you’re rusty on your Greek, “Penta” means “five”. Thus, these scales only have five notes instead of seven–the fourth and seventh notes are dropped from their modal counterparts.

The major and minor pentatonic scales that I learned are truncated versions of the Ionian and Aeolian modal scales, respectively. My guitar teacher also showed me a longer form of both the major and minor pentatonic scales: scales that traverse more frets than the normal four-fret span. Furthermore, the long-form minor scale includes a feature called “The House,” where the notes on the fretboard resemble a house:

Bad rendering of how notes in the minor pentatonic scale resemble a house

“The House” is, apparently, a popular shape used by lots of famous guitar players.

To help me learn these scales, as I have with previous scales, I coded them up in Python in a Jupyter notebook and displayed them in HTML. Here’s the code I wrote:

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
from IPython.display import display_html

%matplotlib inline

major_pentatonic = {'Low E':['','1','','2'], 'A':['3','','','5'], 'D':['6','','','1'], 'G':['2','','3',''], 
                    'B':['','5','','6'], 'High E':['','1','','2']}
minor_pentatonic = {'Low E':['6','','','1'], 'A':['2','','3',''], 'D':['5','','6',''], 'G':['1','','2',''], 
                    'B':['3','','','5'], 'High E':['6','','','1']}
major_pent_long = {'Low E':['1','','2','','3','','',''], 'A':['','','5','','6','','',''], 
                   'D':['','','1','','2','','3',''], 'G':['','','','','5','','6',''], 
                   'B':['','','','','','1','','2'], 'High E':['','','','','3','','','5']}
minor_pent_long = {'Low E':['5','','6','','','','',''], 'A':['1','','2','','3','','',''], 
                   'D':['','','5','','6','','',''], 'G':['','','1','','2','','3',''], 
                   'B':['','','','','','5','','6'], 'High E':['','','','','','1','','2']}

pent_scales = {'Major Pentatonic':major_pentatonic, 'Minor Pentatonic':minor_pentatonic, 
               'Major Pentatonic (Long)':major_pent_long, 'Minor Pentatonic (The House)': minor_pent_long}
pent_html = '<h3>Pentatonic Scale Shapes</h3>'

for i, scale_name in enumerate(pent_scales):
    nbr_of_frets = len(pent_scales[scale_name]['Low E'])
    df_scale = pd.DataFrame(pent_scales[scale_name], index=np.arange(1, nbr_of_frets+1))
    # https://stackoverflow.com/a/50899244
    df_scale_styler = df_scale.style.set_table_attributes("style='display:inline'").set_caption(scale_name)
    pent_html += df_scale_styler._repr_html_() + '&nbsp;&nbsp;&nbsp;'
    if (i+1) % 2 == 0:
        pent_html += '<p></p>'

display_html(pent_html, raw=True)

And that code rendered this graphic:

Pentatonic scales rendered in a Jupyter notebook

So now I can show my notebook in a computer monitor and play along to the scale.

More to come!

Parsing Oddly Formatted Spreadsheets

April 18, 2021 /

Python and pandas works well with conventionally formatted spreadsheets like this:

Conventional spreadsheet easily parsed in Python

But how do you deal with spreadsheets formatted in unconventional ways, like this?

Can you use pandas to parse an oddly formatted spreadsheet?

Here’s my approach to massaging this data into a dataframe I can work with.

Step 1: Go ahead and read in the wonky spreadsheet

Go ahead and read in the spreadsheet, warts and all, into a dataframe. I went ahead and skipped rows 0 and 1 as they were unnecessary:

import pandas as pd

df_raw = pd.read_excel('./data/odd_format.xlsx', skiprows=1).fillna('')

As you’d expect, the results are not very pretty:

Step 2: Figure out where each record starts and stops

Looking at the spreadsheet, I determined that each record starts with a field named “First Name:” and ends with a field named “State:”. If I can put together a list of row indexes that lets me know where each record begins and ends, I should be able to iterate through that list and reformat each record uniformly. Pandas filtering can help with that. To get a list of each “start” row, I can use this code:

df_raw[df_raw['Unnamed: 1']=='First Name:'].index.tolist()

To get a list of each “end” row, I can do this:

df_raw[df_raw['Unnamed: 1']=='State:'].index.tolist()

Finally, I can use Python’s handy zip function to glue both together in a list of tuples that I can easily loop through:

for start_row, end_row in zip(df_raw[df_raw['Unnamed: 1']=='First Name:'].index.tolist(), df_raw[df_raw['Unnamed: 1']=='State:'].index.tolist()):
    # loop through each record

Step 3: Collect all key/value pairs per record

Now that I’m able to iterate over each record, I need to be able to capture each key/value pair in each record: each person’s first name, middle name (if available), last name, etc. I can use Python’s range function to loop from the starting row to the ending row of the record and pandas iloc function to zero in on each key and associated value:

person = {}  # I need some place to store the keys/values, so let's use a dictionary
for i in range(start_row, end_row+1):
    k = df_raw.iloc[i, 1]  # the keys are in column 1
    v = df_raw.iloc[i, 2]  # the values are in column 2

Each record has an empty row in the middle of it, separating “name” properties from “address” properties. I don’t need those empty rows, so I do a quick check before writing the keys and values to my dictionary object:

if len(k.strip()) > 0:
    person[k.strip().replace(':', '')] = v

Of course, I need to be writing each of these person objects to a master list, so I do that by appending each object:

people_list.append(person)

Step 4: Create a new dataframe from the people list

Finally, I can take that clean list of dictionaries and generate a new dataframe from it:

df_clean = pd.DataFrame(people_list)

Which renders a nice dataframe from which I can start my analysis:

That’s a little more like it!

So, putting it all together, my full code looks like this:

import pandas as pd

df_raw = pd.read_excel('./data/odd_format.xlsx', skiprows=1).fillna('')
people_list = []
for start_row, end_row in zip(df_raw[df_raw['Unnamed: 1']=='First Name:'].index.tolist(), df_raw[df_raw['Unnamed: 1']=='State:'].index.tolist()):
    person = {}
    for i in range(start_row, end_row+1):
        k = df_raw.iloc[i, 1]
        v = df_raw.iloc[i, 2]
        
        if len(k.strip()) > 0:
            person[k.strip().replace(':', '')] = v
            
    people_list.append(person)
    
df_clean = pd.DataFrame(people_list)

So, should you encounter similarly unconventionally formatted spreadsheets in the future, hopefully this code will help you find a solution to deal with them!

Learning Guitar with Python, Part 2

January 31, 2021 /

Here’s another installment in my occasional series on using Python to help me learn guitar.

At the start of my practice sessions, I play scales to both help me get warmed up and help me know where the notes in a given key are in relation to each other. I’m learning that scale patterns are important to soloing, so I’m starting to commit a lot of these scales to memory. Having them easily available and all together in a Jupyter Notebook makes this process a lot easier for me.

So, here’s the code I wrote and accompanying image of the modal scales I practice on a regular basis:

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
from IPython.display import display_html

%matplotlib inline


Ionian = {'Low E': ['', '1', '', '2'], 'A': ['3', '4', '', '5'], 'D': ['6', '', '7', '1'],
          'G': ['2', '', '3', '4'], 'B': ['', '5', '', '6'], 'High E': ['7', '1', '', '']}
Dorian = {'Low E': ['', '2', '', '3', '4'], 'A': ['', '5', '', '6', ''], 'D': ['7', '1', '', '2', ''],
          'G': ['3', '4', '', '5', ''], 'B': ['', '6', '', '7', '1'], 'High E': ['', '2', '', '', '']}
Phrygian = {'Low E': ['3', '4', '', '5'], 'A': ['6', '', '7', '1'], 'D': ['2', '', '3', '4'],
            'G': ['5', '', '6', ''], 'B': ['7', '1', '', '2'], 'High E': ['3', '', '', '']}
Lydian = {'Low E': ['', '4', '', '5'], 'A': ['6', '', '7', '1'], 'D': ['2', '', '3', '4'],
          'G': ['5', '', '6', ''], 'B': ['7', '1', '', '2'], 'High E': ['3', '4', '', '']}
Myxolydian = {'Low E': ['', '5', '', '6', ''], 'A': ['7', '1', '', '2', ''], 'D': ['3', '4', '', '5', ''],
          'G': ['6', '', '7', '1', ''], 'B': ['', '2', '', '3', '4'], 'High E': ['', '5', '', '', '']}
Aeolian = {'Low E': ['', '6', '', '7', '1'], 'A': ['', '2', '', '3', '4'], 'D': ['', '5', '', '6', ''],
          'G': ['7', '1', '', '2', ''], 'B': ['', '3', '4', '', '5'], 'High E': ['', '6', '', '', '']}
Locrian = {'Low E': ['7', '1', '', '2'], 'A': ['3', '4', '', '5'], 'D': ['6', '', '7', '1'],
            'G': ['2', '', '3', '4'], 'B': ['', '5', '', '6'], 'High E': ['7', '', '', '']}

modes = {'Ionian':Ionian, 'Dorian':Dorian, 'Phrygian':Phrygian, 'Lydian':Lydian, 'Myxolydian':Myxolydian, 
         'Aeolian':Aeolian, 'Locrian':Locrian}
modes_html = '<h3>Scales for Each Mode</h3>'

for i, mode_name in enumerate(modes):
    nbr_of_frets = len(modes[mode_name]['Low E'])
    df_mode = pd.DataFrame(modes[mode_name], index=np.arange(1, nbr_of_frets+1))
    # https://stackoverflow.com/a/50899244
    df_mode_styler = df_mode.style.set_table_attributes("style='display:inline'").set_caption(mode_name)
    modes_html += df_mode_styler._repr_html_() + '&nbsp;&nbsp;&nbsp;'
    if (i+1) % 3 == 0:
        modes_html += '<p></p>'

display_html(modes_html, raw=True)

This code in a Jupyter Notebook produces this graphic:

The numbers for each note represent the note’s position in the key in which you’re playing. Ionian pushes the first note of the key. For example, if you’re playing in the Key of C major, the “1” would be C. You’d find C on your Low E string (8th fret) and start playing the scale. If you wanted to play the Dorian mode of that key–D Dorian–you’d find D on your Low E string–a whole step up to the 10th fret–and start your Dorian scale there.

When I practice these scales, I’ve been taking a few approaches. For one, I play the scale going up and then turning around and coming back. I’ll start with the first note on the Low E string and play all the way up to the last note on the High E string and then play back down to where I started. To keep things interesting, sometimes I’ll reverse this process so that I start on that last High E string note, play down to the first note on the Low E and then turn around and play back to where I started.

Next, I’ll play each scale for every fret up the neck until I reach the 12th fret. So, for the Ionian scale, I’ll start the scale on the 2nd fret, play it through, move to the 3rd fret, play it through, and repeat this process until I reach the 12th fret. I do this for all the scales, so I play each scale about 10 or so times in a given practice session.

Finally, as I get these scales under my belt, I’m trying to play them faster and faster. I’ll usually try to power through the first four modes as fast and accurate as I can, take a small break, and then finish the last three. Then, I’ll move on to the rest of my practice.

More Python and music notes to follow!

« Older posts Newer posts »

© 2025 DadOverflow.com

Theme by Anders Noren — Up ↑