Musings of a dad with too much time on his hands and not enough to do. Wait. Reverse that.

Month: February 2018 (Page 2 of 2)

10 of My Favorite Free Tools

My dad has countless shop tools to support his mechanical wizardry in the garage. Conversely, I wield a great many software tools to aid my work and interest in technology. Several tools I use are free and others cost some dough. Here’s a list of 10 free tools I find quite useful (in no particular order):

1. Jupyter Notebook

I’ve been teaching myself Python the past 2-3 years and have found Jupyter Notebook an indispensable tool in the endeavor. Jupyter Notebook is basically an integrated development environment (IDE) in your browser. You write your code in “cells”. Console output from a cell will be written right below the code. Even cooler, you can create chart and graph visuals right within your notebook. You can also intersperse your code cells with “markdown” cells. True, developers have a reputation for despising code documentation: nevertheless, interspersing your code cells with markdown cells providing some commentary on what you’re trying to do can make for a neat effect in your notebook–it can certainly remind you of what you were trying to do (if you had to walk away from your code for a while), but it can also turn your entire notebook into a report that you can hand in to management.

2. PowerShell

I’ve talked about PowerShell a few times in the past. I’ve heard people call PowerShell the old Windows DOS shell “on steroids” (what an overused suffix!). If you run Windows 7 or higher, you have PowerShell–so that’s at least 74% of Windows users.  The primary lever in PowerShell is the cmdlet and there are a ton.  Plus, you can write your own. What I find even more compelling, though, is that PowerShell can tap into the entire .NET framework, which gives it tremendous capabilities for a scripting language. If you have some sort of operation you want to automate on your Windows system–from backing up files to pulling down stock closing prices–you can make that happen in PowerShell.

  • Check out some of my PowerShell work here

 

3. Logparser

Logparser is probably one of the more esoteric tools on this list and, frankly, I use it much more at work than I do at home. For the developers, DBAs, and IT operations people out there: imagine if you were able to write everyday SQL, not across a database table but rather across a large, delimited file?  Imagine being about to find the five error messages out of a log file of millions of lines, at the command line, in seconds? Yes, grep can do this, but we’re talking Windows.  And that’s not all: just about all the standard SQL operations are available to Logparser including group, distinct, count, order by, etc. Some basic charting is available, too. I’ve found Logparser to actually parse large files faster than PowerShell, so I’ll often write PowerShell scripts that call Logparser and work with the results. Super combo!

 

4. Slickrun

Slickrun is a tool that probably won’t make a lot of sense until you start forcing yourself to use it. Then, before you know it, you won’t be able to live without it.

Slickrun is like the Windows Run command but on steroids (dangit!).  You start by creating Magic Words: short words or phrases you associate to some action. For instance, I’ve created a “jupyter_notebook” magic word that launches my Jupyter Notebook platform. That way, I don’t have to click the Start Window and scroll through various program files to find the Jupyter Notebook shortcut. Instead, I hit a key combination–in my case, Alt-Q–that opens up the small Slickrun window, then I just start typing “jupyter_notebook” and hit <Enter> to launch. Slickrun even auto-completes magic words, so I tend to be able to launch Jupyter Notebook in about four keyboard strikes–Alt-Q then “ju” as Slickrun will usually auto-complete that to “jupyter_notebook” then <Enter>. I can hit those four keys in a fraction of the time it would take me to navigate the Start Window and find the program. I tie magic words to applications, websites, and even PowerShell scripts and the like to do things like back up my files. Pretty slick, eh?

 

5. Password Safe

Security experts are always telling us to never use the same userid/password combination from one site to another. That way, when your online Dungeons & Dragons account gets pwned, your bank account doesn’t.  But if you’re suppose to have a new userid/password combo for every account you create, how in the world do you keep track of all those credentials?

Well, Password Safe is one way to do it. With Password Safe, you create a strongly encrypted file–your safe. You then protect it with a strong password–the only one you really have to remember. Then, within the safe, you create as many credential entries as you need. You can also create folders within your safe to better organize your credential entries. Furthermore, Password Safe can create strong passwords for you. So, for that next account you create, you can click a “generate” button and have Password Safe create a strong password for you automatically.

In the end, your safe is a file, so make sure you properly back it up.  Sharing your safe across devices can be a challenge, although I believe there are some techniques available to make that easier and there are other products out there like LastPass that focus on solving that problem.

 

6. Visual Studio Code

I still use Microsoft Visual Studio for a number of projects and love JetBrains’ PyCharm, but I’m really trying to embrace Visual Studio Code more and more.

Visual Studio Code is a free IDE developed by Microsoft, of all companies. It supports a great number of programming languages including C#, Python, Java, and even PowerShell. At first, I struggled to learn my way around the tool, but I’m starting to find an increasing number of tutorials and presentations that use it, so that’s helpful. All the cool kids seem to be using VS Code, so it’s probably a good thing to add to your toolbox.

 

7. Notepad++

Notepad++ has been around for a while, but I continue to use it everyday. While I also use Microsoft Notepad for simple tasks, nothing beats text editing in Notepad++ with its multi-tab interface and plugin support. I find it very helpful for formatting XML and JSON files, using its regex find/replace features, and even using XPath query operations on occasion.

 

8. Paint.net

I’ve known about Paint.net for a while, but have only started using it recently to build logos and images like the one at the top of this page. I am absolutely no graphic artist and Paint.net’s interface can be quite intimidating, but it can help you craft some pretty nifty images. I’ve scoured YouTube for as many tutorials as I can find to try to shorten my learning curve with the product–I recommend this one in particular for making logos.

 

9. Git-Bash

Git is what all the cool kids do for software source control and, since all the cool kids work frequently from the command line, Git Bash is what you need for all your source code management operations. Aside from git utilities, you also get a fair amount of Bash utilities–like a two-for-one special! In the past, I used Cygwin to get a Bash experience on my Windows machines, but, going forward, I’m going to try to perform all the Bash-based work I need to do in Git Bash, instead, and see where that gets me.

 

10. Q-dir

Pop quiz: how many Windows Explorer instances do you have open right now? How many of them tend to stay open? Q-dir stands for “Quad Directory” or “Quad Explorer”. By default, Q-dir is a single window split into four sections. Each of the four sections hosts its own Windows Explorer instance. So, off the bat, you have four File Explorers in one window. Even cooler, each “mini explorer” can have tabs that point to other directory paths. Pretty awesome! So, for the most part, you don’t need to have five instances of Windows Explorer running and then try to rifle through each instance of find the one you’re looking for: instead, run one instance of Q-dir and find the Windows Explorer you need there.

 

So, that’s just a small taste of many of the free tools I use to try to make better use of my time. If you really want to geek out on tools, I highly recommend Scott Hanselman’s Ultimate Tools guide.

Working around Excel

Every-so-often I find myself having to do some processing with a Microsoft Excel file–grouping data, performing different statistical operations, etc. Doing much of this work within Excel itself is just a pain, so I would always open the file and save the applicable worksheets to CSV files.  But even manually saving the worksheets to CSV was a pain, so I devised a way to do the whole thing with PowerShell.

Unfortunately, PowerShell still has no convenient Convert-ExcelToCSV cmdlet…so I wrote my own. For added convenience, I built my function as a module and even added an Import-Module statement in my profile. Here are some quick steps on how I did that:

  1. Under your %userprofile%\documents directory, make sure you have a WindowsPowerShell directory. If not, create one.
  2. Under that WindowsPowerShell directory, create a modules directory, if one doesn’t already exist.
  3. Finally, under the modules directory, create a directory called Convert-ExcelToCSV.
  4. In that Convert-ExcelToCSV directory, drop a copy of my Convert-ExcelToCSV.psm1 module.
  5. Back under the WindowsPowerShell folder, create a file named Microsoft.PowerShell_profile.ps1.
  6. Edit Microsoft.PowerShell_profile.ps1 and add the statement Import-Module Convert-ExcelToCSV

(For more on PowerShell profiles, check out this Scripting Guy post)

So, that’s pretty cool. Now, you can open up a PowerShell command shell and save your Excel files to CSV in a one-liner:

Convert-ExcelToCSV -FileName "C:\some\path\my_excel_file.xlsx"

But we can still make this even more convenient: how about, when I right-click on an Excel file, I get the option to convert the sucker to CSV right there? Well, we can do that by editing the Windows Registry.

According to my research, in order to cover the older XLS file versions as well as the newer XLSX versions, you have to edit two slightly different paths under the HKEY_CLASSES_ROOT hive. Check out my registry file for enabling this feature for the older file type and my other file for enabling this feature on new versions of Excel files.

To make these registry modifications, you can download my REG files and run them on your machine.

Warning: editing your Registry is a big deal. I’ve tested these on my system and they work, but make sure you review the modifications first and make sure you’re comfortable with everything before proceeding.

With these modifications, you can now convert your Excel files to CSVs with ease.

 

Python makes it even easier

So, I had to do a lot of work to get to a point where I can easily convert my Excel files to CSVs for easier processing down the line. Well, guess what? Python makes it even simpler with pandas.  Check out the function read_excel.  That is just awesome!

So, either way you go–PowerShell or Python–you have options to free yourself from some of the frustrations of Excel!

Newer posts »

© 2024 DadOverflow.com

Theme by Anders NorenUp ↑