Let’s look at a quick real-world example of how you can use this.įirstly let’s install the tool using pip install. The tasks that it can execute can be divided into three areas: input, processing and output. curl -o weather.json -X GET \ '' \ -H 'Postman-Token: dcf3c17f-ef3f-4711-85e1-c2d928e1ea1a' \ -H 'cache-control: no-cache' csvkitĬsvkit is a set of command line tools for working with CSV files. For a more comprehensive tutorial on cURL see this excellent article by Zaiste. Running the following command will obtain a single record from the OpenWeatherMap API and save as a JSON file named weather.json. This tool also supports both GET and POST requests for interacting with an API. curl -o data_dl.csv Īnother common method of obtaining data for data science projects is via an API. The file will now be available in your current working directory. If we simply run curl which in our example will be curl this will print the data to the terminal.Īdding some additional arguments will download and save the data using a specified filename.
I am going to use a simple curl command to download a data set taken from the blood transfusion centre in Hsin-Chu City, Taiwan. The UCI Machine Learning Repository is an excellent resource for obtaining datasets for machine learning projects. I’ll give a couple of example use cases for obtaining publically available data sets. This is a useful tool for obtaining data from any server via a variety of protocols including HTTP. It turns out there are many tasks that can be completed via simple terminal commands than I first thought and I wanted to share some of those here. In the following post, I am going to list some of the best tools I have found for doing data science on the command line. Sometimes it is nice to find ways of doing more things in the same piece of software.
#COMMAND LINE PHOTO RENAMER DATE TAKEN CODE#
You can be editing some code in a Jupyter Notebook, having to install a new tool on the command line and maybe editing a function in an IDE all whilst working on the same task. One of the most frustrating aspects of data science can be the constant switching between different tools whilst working.
0 kommentar(er)
