goodls

<a name="top"></a>

<a name="overview"></a>

Overview

This is a CLI tool to download shared files and folders from Google Drive. For large file, the resumable download can be also run.

<a name="description"></a>

Methods

1. Download shared files from the shared URL without the authorization.

We have already known that the shared files on Google Drive can be downloaded without the authorization. But when the size of file becomes large (about 40MB), it requires a little ingenuity to download the file. It requires to access 2 times to Google Drive. At 1st access, it retrieves a cookie and a code for downloading. At 2nd access, the file is downloaded using the cookie and code. I created this process as a CLI tool.

2. Download all shared files with the folder structure from the shared folder.

This method uses API key.

There are sometimes the situation for downloading files in a shared folder. But I couldn't find the CLI applications for downloading files in the shared folder. So I implemented this. But when in order to retrieve the file list from the shared file, Drive API is required to be used. In order to use Drive API, it is required to use OAuth2, Service account and API key. So I selected to use API key which is the simplest way. This CLI tool can retrieve the file list in the shared folder using API key and download all files in the shared folder. From version 1.2.2, you can retrieve only files with the specific mimeType from the folder.

3. Run resumable download for large files.

This method uses API key.

At a recent proposal, I knew the requirement of the resumable download of shared file. So I implemented this.

How to Install

Download an executable file of goodls from the release page and import to a directory with path.

or

Use go get.

$ go install github.com/tanaikech/goodls@latest

• GO111MODULE=on

Usage

<a name="downloadsharedfiles"></a>

1. Download shared files

<a name="demo1"></a>

The image used for this demonstration was created by k3-studio

<a name="demo2"></a>

You can use this just after you download or install goodls. You are not required to do like OAuth2 process.

$ goodls -u [URL of shared file on Google Drive]

• Help
  • $ goodls --help
• Options
  • -e
    • Extension of output file. This is for only Google Docs (Spreadsheet, Document, Presentation). Default is pdf. When ms is used, the shared Google Docs can be downloaded as Microsoft Docs.
    • Sample :
      • $ goodls -u https://docs.google.com/document/d/#####/edit?usp=sharing -e txt
  • -f
    • Filename of file which is output. When this was not used, the original filename on Google Drive is used.
    • Sample :
      • $ goodls -u https://docs.google.com/document/d/#####/edit?usp=sharing -e txt -f sample.txt
• URL is like below.
  • In the case of Google Docs (Spreadsheet, Document, Slides)
    • https://docs.google.com/spreadsheets/d/#####/edit?usp=sharing
    • https://docs.google.com/document/d/#####/edit?usp=sharing
    • https://docs.google.com/presentation/d/#####/edit?usp=sharing
  • In the case of except for Google Docs
    • https://drive.google.com/file/d/#####/view?usp=sharing
  • In the case of webContentLink
    • https://drive.google.com/uc?export=download&id=###

File with several URLs

If you have a file including URLs, you can input the URL data using standard input and pipe as follows. If wrong URL is included, the URL is skipped.

$ cat sample.txt | goodls

or

$ goodls < sample.txt

sample.txt

https://docs.google.com/spreadsheets/d/#####/edit?usp=sharing
https://docs.google.com/document/d/#####/edit?usp=sharing
https://docs.google.com/presentation/d/#####/edit?usp=sharing

When you download shared files from Google Drive, please confirm whether the files are shared.

<a name="downloadfilesfromfolder"></a>

2. Download all files from shared folder

<a name="demo3"></a>

When above structure is downloaded, the command is like below. At that time, the folder ID is the folder ID of "sampleFolder1".

Files are downloaded from the shared folder. In this demonstration, the fake folder ID and API key are used.

<a name="retrieveapikey"></a>

Retrieve API key

In order to use this, please retrieve API key as the following flow.

1. Login to Google.

2. Access to https://console.cloud.google.com/?hl=en.

3. Click select project at the right side of "Google Cloud Platform" of upper left of window.

   • If you cannot see the project, please try to access to https://console.cloud.google.com/cloud-resource-manager?hl=en. You can also create new project at there. - When you created new project there, please click the left of "Google Cloud Platform". You can see it like 3 horizontal lines. By this, a side bar is opened. At there, please select "API & Services" -> "Library". After this, please check the following flow from Click "Library" at left side..

4. Click "NEW PROJECT"

   1. Input "Project Name".
   2. Click "CREATE".
   3. Open the created project.
   4. Click "Enable APIs and get credentials like keys".
   5. Click "Library" at left side.
   6. Input "Drive API" in "Search for APIs & Services".
   7. Click "Google Drive API".
   8. Click "ENABLE".
   9. Back to https://console.cloud.google.com/?hl=en.
   10. Click "Enable APIs and get credentials like keys".
   11. Click "Credentials" at left side.
   12. Click "Create credentials" and select API key.
   13. Copy the API key. You can use this API key.

Registering API key to environment variable

• When API key is registered to the environment variable. When GOODLS_APIKEY as the environment variable is set API key, goodls uses API key from the environment variable.
• If API key is used with the option at the command line, the priority of option is higher than the environment variable.

For example, in the case of bash, please add a following code to .bashrc.

export GOODLS_APIKEY=### your API key ###

Download

When the URL of shared folder is https://drive.google.com/drive/folders/#####?usp=sharing, you can download all files in the folder by the following command.

$ goodls -u https://drive.google.com/drive/folders/#####?usp=sharing -key [APIkey]

• Project files cannot be downloaded by API key. If you want to download the project files, you can download them by ggsrun, because ggsrun uses OAuth2.
• This new function uses the Go library of go-getfilelist.
• When the option of --NoProgres, -np is used, the progress information is not seen. This is a silent mode.
• If the files which are tried to be downloaded are existing, an error occurs. But when you use the option --overwrite and --skip, the files are overwritten and skipped, respectively.
• If you use the option -m, you can retrieve only files with the specific mimeType from the folder. For example, when you want to download PDF and PNG files, please run like below.
  • $ goodls -u [URL] -key [APIkey] -m "application/pdf,image/png"

Retrieve information of file and folder

When you want to retrieve the information of file and folder, you can do it as follows.

For file

$ goodls -u https://docs.google.com/spreadsheets/d/#####/edit?usp=sharing -key [APIkey] -i

For folder

$ goodls -u https://drive.google.com/drive/folders/#####?usp=sharing -key [APIkey] -i

<a name="resumabledownloadoffile"></a>

3. Resumable download of shared file

When you use this option, at first, please retrieve API key. About how to retrieve API key, you can see at here.

When you want to download 100 MBytes of the shared file, you can use the following command.

$ goodls -u [URL of shared file on Google Drive] -key [APIkey] -r 100m

• Please use the option -r. In this sample, 100m means to download 100 MBytes of the shared file.
  • If you want to download 1 GB, please use -r 1g.
  • If you use -r 1000000, 1 MByte of the file will be able to be downloaded.

You can see the actual running of this option at the following demonstration movie.

<a name="demo4"></a>

In this demonstration, the following command is run 3 times.

$ goodls -u https://drive.google.com/drive/folders/abcdefg?usp=sharing -key htjklmn -r 80m

• At 1st run, the data of 0 - 80 MBytes is downloaded.
  • You can see New download at "Current status".
• At 2nd run, the data of 80 - last is downloaded.
  • You can see Resumable download at "Current status".
• At 3rd run, the download has already been done. So the checksum is shown.
  • You can see Download has already done. at "Current status".

Note

• Reason that API key is used for this.
  • When it accesses to the shared file without the authorization, the file size and md5checksum cannot be retrieved. So in order to use Drive API, I adopted to use API key.
• Reason that the download size is inputted every time.
  • When this option is run 1 time, 1 quota is used for Drive API. So I adopted this way.

Q&A

• I