DadOverflow.com

Musings of a dad with too much time on his hands and not enough to do. Wait. Reverse that.

Month: August 2019

Teaching the kid to drive

August 17, 2019 /
Patience is a must

I’m now on to teaching my second child to drive. Here are some thoughts I’ve accumulated over the time that might be of some assistance to you novice parent instructors out there:

Review the car mechanics

Your child should obviously know the gas pedal from the brake, but don’t forget all the other knobs and switches that we know by heart but your child doesn’t. Gear shift, turn signals, headlights, bright lights, windshield wipers/fluid, parking brake, hazard lights…make sure to review all these features and how to activate them. As an advanced topic, I show my child how to halfway hold the turn signal arm up or down–thus, engaging the turn signal but not locking the arm in place, which can be handy for short-duration signaling like changing lanes.

Your safety radar

I try to get my kids to imagine they’re detectives or air traffic controllers. They need to be looking all around them, looking for new threats, obstacles, and other information appearing on their radars; they should be looking for clues to let them know how these objects might cause safety concerns. Is a given car in the vicinity driving erratically, not using turn signals, or otherwise providing evidence of potential danger? Add that car to your radar and keep an eye in it–and some distance. Are there children nearby that have the potential of running in front of you? Add them to your radar and take caution.

The first lesson: the parking lot

It’s probably a no-brainer, but my first lesson with the kids is in a large, unpopulated parking lot. In fact, the first several lessons will be in said parking lot. I have them drive in circles, clockwise and counter-clockwise. Several lessons in, I will start having them drive in reverse–maybe even doing a full circle or two in reverse. Of course, I also have the kids practice parking: both nose-first and rear-first. In every lesson, I insist we pretend that the parking lot is full and my young driver is not allowed to hit any imaginary car.

Residential Driving

Lazy suburbs are ideal starting points for getting your child on a real road. Hopefully, your chosen suburb has limited cars parked on the side of the roads. Initially, you may want to target early mornings or evenings to limit pedestrian traffic, as well. One alternative to the suburb is to practice on the access roads of a medium-sized cemetery.

U.S. Routes and State Highways

Once my child seems comfortable driving on the suburbs and local roads, we graduate to the nearby U.S. route. U.S. routes are a fantastic resource for beginner drivers. They traverse a variety of driving conditions: small town driving at 25-35 mph, mid-town driving at 40-45 mph, and even highway driving at 60+ mph. State routes have stop sign intersections, stoplights, railroad crossings, and multi-lane roadways. They’re also great to log lots of drive-time hours. Often, the kid and I will get up early on a Saturday or Sunday, drive two hours out then turn around and come back home.

Interstate Highway Driving

Interstate highway driving is almost like a final exam for my kids. Things get real on the highways. Obviously, the less traffic the better when starting out on a highway, so early mornings and even the weekday gaps between rush hours seem to work well. Here’s one technique I heard of recently that I may incorporate in my future lessons: drive to a convenient beltway, then proceed to take the exit. From the exit, work your way over to the closest entrance ramp to get right back on the beltway. Rinse and repeat at every exit until you’ve made your way around the entire highway. That’s sure to train your child up on highway exits and entrances.

Nighttime Driving

Repeat all the above, but this time, at night.

Odds and Ends

Don’t forget some of these other aspects of driving:

  • Gassing up the car (or charging, as the case may be)
  • Parking in full parking lots
  • Drive-up ATMs and fast food drive-throughs
  • What to do if you have an accident or other emergency
  • How to read your dashboard and where to look for errors and warnings
  • How to listen to the sound of the car under normal operating conditions and how that sound might change if the car is malfunctioning
  • Parallel parking?!

So, those are a few of the items I work through with my children as I teach them to drive. Any ideas I missed? Feel free to add them in the comments!

How do you transpose a Question/Answer dataset?

August 12, 2019 /

Recently, a friend came to me with an interesting challenge. He had a dataset of questions and answers where each record contained a single question and the answer to the question. Arguably, this dataset was already in a tidy format, but my friend wanted to transpose the data such that each unique question became a column of its own with the answers as values.

Before I could come to his aid, my friend already found a great answer at Medium.com using the pandas function: pivot_table.

Here’s what he did:

Let’s suppose you have this table of question/answer, tab-delimited data:

person	question	answer
Sir Robin	What is your name?	Sir Robin of Camelot
Sir Robin	What is your quest?	To seek the Holy Grail
Sir Robin	What is the capital of Assyria?	I don't know that
Sir Lancelot	What is your name?	Sir Lancelot of Camelot
Sir Lancelot	What is your quest?	To seek the Holy Grail
Sir Lancelot	What is your favorite colour?	Blue
Sir Galahad	What is your name?	Sir Galahad of Camelot
Sir Galahad	What is your quest?	I seek the Grail
Sir Galahad	What is your favorite colour?	"Blue, no Yellow"
King Arthur	What is your name?	"Arthur, King of the Britons"
King Arthur	What is your quest?	I seek the Holy Grail
King Arthur	What is the air speed of an unladened swallow?	What do you mean?  An African or European swallow?

Step 1: Import pandas and read in your data

import pandas as pd

df = pd.read_csv('questions.csv', sep='\t')

Step 2: pivot_table

df_pivotted = df.pivot_table(index='person', values=['answer'], 
                             columns=['question'], aggfunc=lambda x: ' '.join(str(v) for v in x))
df_pivotted.head()
pivot_table does the job nicely

The trick here is the aggfunc operation. The aggfunc parameter is normally used to sum, average, or perform some other type of numeric operation on your values columns. Interestingly, though, you can apparently supply your own custom function to this parameter instead. Here, the Medium.com author found that he could simply loop through every letter of the answer and re-join them with spaces, effectively return the original answer.

That seems pretty complicated

The use of pivot_table certainly works in this example and it’s pretty sweet to see that you can pass your own custom function to it. However, pandas also has a more generic, pivot function. Could that have worked here?

The answer is: yes. When you google pandas pivot vs pivot_table, one of the top responses is this Stackoverflow.com post that suggests pivot_table only allows numerically-typed columns in the values parameter while pivot will take strings. I don’t think this is quite true, since the above example passed a string column to the values parameter, but it does suggest that pivot might be more disposed to working with strings than pivot_table. Let’s give it a try:

df.pivot(index='person', values='answer', columns='question')
Whaddya know?! Pivot can do the job, too!

Not only can pivot do the transformation, it certainly seems less complicated. Check out my full code here.

How do you hide secrets in Jupyter Notebooks?

August 9, 2019 /

Often in my notebooks, I will connect to a relational database or other data store, query the system for data, and then do all sorts of amazing operations with said data. Many times, these data stores are restricted to select users and I must authenticate myself to the system–usually with an id and password. One might be inclined to code such connection strings inline in his Jupyter Notebook. However, I usually check my notebooks in to source control and/or hand them in to management as reports or documentation. Thus, any number of people might see my notebooks potentially compromising my personal id and password were I to code the credentials inline.

So, how can I hide my secrets–my connection strings and other sensitive information–so I can still safely share the good work I do in my notebooks? The way I do it is by moving my connection strings to configuration files. Allow me to demonstrate:

Step 1: Import my packages

from sqlalchemy import create_engine
import pandas as pd
from configparser import ConfigParser

I import the usual suspects–SQLAlchemy for database management and pandas for my dataframe work–but I’m also loading in configparser. It’s this last package that will help me pull out my secret stuff to a separate file that I can protect.

Step 2: Create my configuration file

Now, I need to create that separate configuration file. In the same directory as my notebook, I’ll create a text file. I usually name my file nb.cfg–as in, notebook config. For my example, storing the connection string to my SQLite database, my configuration file looks like so:

[my_db]
conn_string: sqlite:///mwc.db

Although SQLite databases don’t have authentication requirements, you can imagine, say, a connection string to a PostgreSQL database that would contain an id and password.

Step 3: Load the configuration file

Back in your notebook, load your configuration file:

parser = ConfigParser()
_ = parser.read('nb.cfg')

Step 4: Access the secrets in your configuration file

Now you’re ready to access those secrets! In this example, I’ll pass my secret connection string to my database engine object:

engine = create_engine(parser.get('my_db', 'conn_string'))

Step 5: Profit!

That’s basically it. In my example, I can now use my database engine object to query a table in my database and load the results into a dataframe:

qry = """
SELECT *
FROM people
"""

df_mwc_people = pd.read_sql(qry, engine)

Check out the complete code example here.

Postscript

You might ask yourself, “self, do I need to do anything else to protect my config file from getting into the hands of my enemies?” Well, since I often use Git for source control, I do want to make sure I don’t accidentally check my configuration file into my source code repository. To avoid that problem, I create a .gitignore file and add the name of my configuration file to it. Then, every time I commit a change, Git will simply ignore committing my configuration file.

© 2024 DadOverflow.com

Theme by Anders Noren — Up ↑