https://eblog.ink/category/%E7%BD%91%E7%9B%98%E7%9B%B8%E5%85%B3/ https://eblog.ink/archives/245/ 2020-09-01T06:00:00+00:00 1.准备工作：谷歌服务器一台 域名一个并NS到CF 创建一个TG机器人（BotFather ） 团队盘一个 2.域名通过CF解析到服务器ssl改为灵活自动代理3.使用一键安装部署脚本一键脚本使用方法：只需复制以下链接到VPS命令行窗口粘贴回车即可以执行gdutils项目一键部署脚本（包括“查询转存”和“TG机器人”两部分）bash -c "$(curl -fsSL https://raw.githubusercontent.com/Eric-Qiu1994/gd-utils/master/gdutilsinstall.sh)" gdutils项目一键部署脚本之“转存查询部分”bash -c "$(curl -fsSL https://raw.githubusercontent.com/Eric-Qiu1994/gd-utils/master/gdutilscsinstall.sh)" gdutils项目一键部署脚本之“TG机器人部分”bash -c "$(curl -fsSL https://raw.githubusercontent.com/Eric-Qiu1994/gd-utils/master/gdutilsbotinstall.sh)" 4.获取SAsudo git clone https://github.com/Eric-Qiu1994/AutoRclone && cd AutoRclone && sudo pip3 install -r requirements.txt 5.生成service accounts开启Drive API如果你之前没创建过项目, 直接运行python3 gen_sa_accounts.py --quick-setup 5 创建6个项目（项目0到项目5）开启相关的服务创建600个service accounts（6个项目，每个项目100个）将600个service accounts的授权文件下载到accounts文件夹下面如果你已经有N个项目，现需要创建新的项目并在新的项目中创建service accounts，直接运行python3 gen_sa_accounts.py --quick-setup 2 --new-only 额外创建2个项目（项目N+1到项目N+2）开启相关的服务创建200个service accounts（2个项目，每个项目100个）将200个service accounts的授权文件下载到accounts文件夹下面如果你想用已有的项目来创建service accounts（不创建新的项目），直接运行python3 gen_sa_accounts.py --quick-setup -1 注意这会覆盖掉已有的service accounts顺利完成后，AutoRclone文件下面的accounts文件夹下会有很多的json文件。如果遇到问题进项目管理点击后面的三个点儿，选择设置，然后把每一个项目里面的服务号全部删掉 项目管理连接把生成的这些.json文件复制到sa目录6.把Json文件转换为邮箱https://raw.githubusercontent.com/Eric-Qiu1994/A-python-script-of-AutoRclone-or-gclone/master/mail.py 通过python3转换为txt7.创建谷歌群组谷歌群组连接把邮箱加入到新建的谷歌群组内（每天限制加入100个）注意：在 gd-utils 目录下，执行 git pull 拉取最新代码，如果你使用了 pm2 守护进程，执行pm2 reload server刷新生效 https://eblog.ink/archives/244/ 2020-08-30T02:09:00+00:00 Demo，不放。网站下载链接直接是xxx.com/abc.jpg，没有302跳到微软xxx.sharepoint.com，也没有反代xxx.sharepoint.com那样后面一长串。准备工作：1， Onedrive，或Sharepoint这个大家有的，各种A1，E3，E5，世纪。（Sharepoint默认25T）2， OneManager，Github地址：https://github.com/qkqpttgf/OneManager-php基于各种无服务器的平台，比如Heroku、腾讯SCF、阿里FC、华为FG、百度CFC，配置都保存在其环境变量中，当然也可以装在VPS与空间里，不过为了一致，程序不固定储存onedrive的目录结构，缓存最多1小时3， Heroku，最好需要注册2个帐号，注册与登录时需要魔法，平时操作时不需要OneManager配置是保存在环境变量中的，所以不需要一直监控保活，Heroku免费套餐每月550小时，550小时/30天=18.33小时，每天只要访问时间段空出来24-18=6小时没有人访问，这一个月其实就足够使用了，但有些人的网站可能访问人比较多所以还没到月底就用光了。当然，heroku也可绑卡达到1000小时，同时还能自定义域名。但，SSL却比较麻烦。4， CF workers，免费的就好部署程序、安装程序、绑定网盘：1， 将OneManager一键部署到Heroku2， 访问给定的herokuapp域名，正式安装好程序3， 安装好后登录，在设置中绑定Onedrive或Sharepoint4， 在设置中做些设定5， 换一个Heroku帐号再安装一次（时长够用就无所谓）重点：在CF Workers中填入以下代码// 单日 const SingleDay = '111.herokuapp.com' // 双日 const DoubleDay = '222.herokuapp.com' // 一切给CF代理，true/false const CFproxy = true // 由于heroku不绑卡不能自定义域名，就算绑卡后https也不方便 // 另外免费套餐每月550小时，有些人不够用 // 于是在CF Workers使用此代码，分单双日拉取不同heroku帐号下的相同网页 // 只改上面，下面不用动 addEventListener('fetch', event => { let nd = new Date(); if (nd.getDate()%2) { host = SingleDay } else { host = DoubleDay } if (!CFproxy) { let url=new URL(event.request.url); if (url.protocol == 'http:') { url.protocol = 'https:' response = Response.redirect(url.href); event.respondWith( response ); } else { url.hostname=host; let request=new Request(url,event.request); event.respondWith( fetch(request) ) } } else { event.respondWith( fetchAndApply(event.request) ); } }) async function fetchAndApply(request) { let response = null; let url = new URL(request.url); if (url.protocol == 'http:') { url.protocol = 'https:' response = Response.redirect(url.href); return response; } url.host = host; let method = request.method; let body = request.body; let request_headers = request.headers; let new_request_headers = new Headers(request_headers); new_request_headers.set('Host', url.host); new_request_headers.set('Referer', request.url); let original_response = await fetch(url.href, { method: method, body: body, headers: new_request_headers }); response = new Response(original_response.body, { status: original_response.status, headers: original_response.headers }) return response; } 单双日分别填入不同的heroku帐号下的网站（当然2个网站要设置得一模一样），这样每个heroku帐号每个月都只访问15或16天，就算24小时有人访问也才360或384小时。最后，给workers加上自定义域名的路由！从头到尾都可以是Free的，不看网站页面，光从下载过程来看，可以完全假装CF套在了大盘鸡上,谁也看不出我没服务器.原文链接 https://eblog.ink/archives/234/ 2020-07-10T02:45:00+00:00 Google 虽然是大厂，但是数据只有一份，还是怕怕，多一个备份不是坏事。苹果 iCloud 不错，但是贼贵，况且我只有一个进过水的 SE……那就 OneDrive 吧，Office 365 就有 1T，前阵子在老司机群看到博友合租 Office 365 家庭套餐，还挺划算的，于是就上车了。1.1 rclone 安装执行命令：curl https://rclone.org/install.sh | sudo bash1.2 连接网盘连接 Google Drive，执行命令进入配置菜单：rclone config1.3 初次进入会有如下菜单提示：#~:rclone config No remotes found - make a new one n) New remote -> 连接新远程网盘 s) Set configuration password -> 设置密码保护配置信息 q) Quit config -> 退出 n/s/q> 1.4 需要加密码保护配置信息就按 s 进入设置密码，若不需要，按 n 进入连接网盘。n/s/q> n name> gdrive -> 网盘名字，写一个方便自己识别的名字就好 Type of storage to configure. Enter a string value. Press Enter for the default (""). Choose a number from below, or type in your own value 1 / Alias for a existing remote \ "alias" 2 / Amazon Drive \ "amazon cloud drive" 3 / Amazon S3 Compliant Storage Providers (AWS, Ceph, Dreamhost, IBM COS, Minio) \ "s3" 4 / Backblaze B2 \ "b2" 5 / Box \ "box" 6 / Cache a remote \ "cache" 7 / Dropbox \ "dropbox" 8 / Encrypt/Decrypt a remote \ "crypt" 9 / FTP Connection \ "ftp" 10 / Google Cloud Storage (this is not Google Drive) \ "google cloud storage" 11 / Google Drive \ "drive" 12 / Hubic \ "hubic" 13 / JottaCloud \ "jottacloud" 14 / Local Disk \ "local" 15 / Mega \ "mega" 16 / Microsoft Azure Blob Storage \ "azureblob" 17 / Microsoft OneDrive \ "onedrive" 18 / OpenDrive \ "opendrive" 19 / Openstack Swift (Rackspace Cloud Files, Memset Memstore, OVH) \ "swift" 20 / Pcloud \ "pcloud" 21 / QingCloud Object Storage \ "qingstor" 22 / SSH/SFTP Connection \ "sftp" 23 / Webdav \ "webdav" 24 / Yandex Disk \ "yandex" 25 / http Connection \ "http" Storage> 11 -> 11 是 Google Drive Google Application Client Id Leave blank normally. Enter a string value. Press Enter for the default (""). client_id> -> 留空，按回车下一步就好 Google Application Client Secret Leave blank normally. Enter a string value. Press Enter for the default (""). client_secret> -> 留空，按回车下一步就好 Scope that rclone should use when requesting access from drive. Enter a string value. Press Enter for the default (""). Choose a number from below, or type in your own value 1 / Full access all files, excluding Application Data Folder. \ "drive" 2 / Read-only access to file metadata and file contents. \ "drive.readonly" / Access to files created by rclone only. 3 | These are visible in the drive website. | File authorization is revoked when the user deauthorizes the app. \ "drive.file" / Allows read and write access to the Application Data folder. 4 | This is not visible in the drive website. \ "drive.appfolder" / Allows read-only access to file metadata but 5 | does not allow any access to read or download file content. \ "drive.metadata.readonly" scope> 2 -> 1 是拥有所有权限，包括删除；2 是只能访问所有文件，不能删除，按需要选择吧 ID of the root folder Leave blank normally. Fill in to access "Computers" folders. (see docs). Enter a string value. Press Enter for the default (""). root_folder_id> -> 留空，按回车下一步就好 Service Account Credentials JSON file path Leave blank normally. Needed only if you want use SA instead of interactive login. Enter a string value. Press Enter for the default (""). service_account_file> -> 留空，按回车下一步就好 Edit advanced config? (y/n) y) Yes n) No y/n> n -> 一般不需要高级设置，选择 n 就好 Remote config Use auto config? * Say Y if not sure * Say N if you are working on a remote or headless machine or Y didn't work y) Yes n) No y/n> n -> 如果是在 VPS 上连接，选 n 吧 If your browser doesn't open automatically go to the following link: https://accounts.google.com/o/oauth2/auth?access_type=offline&client_id=************************************** Log in and authorize rclone for access Enter verification code> **************zN5****vd5*******************bwZethESs Configure this as a team drive? y) Yes n) No y/n> n -------------------- [remote] client_id = client_secret = scope = drive root_folder_id = service_account_file = token = {"access_token":"XXX","token_type":"Bearer","refresh_token":"XXX","expiry":"2018-09-16T13:57:58.955387075Z"} -------------------- y) Yes this is OK e) Edit this remote d) Delete this remote y/e/d> y 1.5 Google Drive 连接完成，可以执行命令检查一下是否真的能连接上：#列出顶级文件夹，（gdrive 是前面第一步设置的名字，如果配置设置了密码会提示输入密码） rclone lsd gdrive: #列出所有文件 rclone ls gdrive: 1.6 连接 OneDrive，方法类似，命令进入配置：rclone config#~:rclone config No remotes found - make a new one n) New remote s) Set configuration password q) Quit config n/s/q> n name> onedrive Type of storage to configure. Enter a string value. Press Enter for the default (""). Choose a number from below, or type in your own value 1 / Alias for a existing remote \ "alias" 2 / Amazon Drive \ "amazon cloud drive" 3 / Amazon S3 Compliant Storage Providers (AWS, Ceph, Dreamhost, IBM COS, Minio) \ "s3" 4 / Backblaze B2 \ "b2" 5 / Box \ "box" 6 / Cache a remote \ "cache" 7 / Dropbox \ "dropbox" 8 / Encrypt/Decrypt a remote \ "crypt" 9 / FTP Connection \ "ftp" 10 / Google Cloud Storage (this is not Google Drive) \ "google cloud storage" 11 / Google Drive \ "drive" 12 / Hubic \ "hubic" 13 / JottaCloud \ "jottacloud" 14 / Local Disk \ "local" 15 / Mega \ "mega" 16 / Microsoft Azure Blob Storage \ "azureblob" 17 / Microsoft OneDrive \ "onedrive" 18 / OpenDrive \ "opendrive" 19 / Openstack Swift (Rackspace Cloud Files, Memset Memstore, OVH) \ "swift" 20 / Pcloud \ "pcloud" 21 / QingCloud Object Storage \ "qingstor" 22 / SSH/SFTP Connection \ "sftp" 23 / Webdav \ "webdav" 24 / Yandex Disk \ "yandex" 25 / http Connection \ "http" Storage> 17 Microsoft App Client Id Leave blank normally. Enter a string value. Press Enter for the default (""). client_id> Microsoft App Client Secret Leave blank normally. Enter a string value. Press Enter for the default (""). client_secret> Edit advanced config? (y/n) y) Yes n) No y/n> n Remote config Choose OneDrive account type? * Say b for a OneDrive business account -> b 商业版 * Say p for a personal OneDrive account -> p 个人版 b) Business p) Personal b/p> p Use auto config? * Say Y if not sure * Say N if you are working on a remote or headless machine y) Yes n) No y/n> n For this to work, you will need rclone available on a machine that has a web browser available. Execute the following on your machine: rclone authorize "onedrive" Then paste the result below:（需要在能打开浏览器的电脑上获取，方法见下面：OneDrive 授权码获取方法） result>{"access_token":"$$$$$$$$$$$$$$","expiry":"2018-09-03T08:43:55.5290909+08:00"} 2018/09/03 07:47:11 ERROR : Failed to save new token in config file: section 'onedrive' not found -------------------- [onedrive] type = onedrive token = {"access_token":"$$$$$$$$$$$$$$","expiry":"2018-09-01T08:43:55.5290909+08:00"} -------------------- y) Yes this is OK e) Edit this remote d) Delete this remote y/e/d> y Current remotes: Name Type ==== ==== onedrive onedrive e) Edit existing remote n) New remote d) Delete remote r) Rename remote c) Copy remote s) Set configuration password q) Quit config e/n/d/r/c/s/q> q 1.7 OneDrive 授权码获取方法：官网下载 rclone Windows 版，地址：https://rclone.org/downloads/如 Rclone Download v1.43 Windows 64 位：https://downloads.rclone.org/v1.43/rclone-v1.43-windows-amd64.zip，解压到 C 盘根目录，文件夹重命名为 rc，然后参考下面方法即可获取授权。Microsoft Windows [版本 10.0.14393] (c) 2016 Microsoft Corporation。保留所有权利。 C:\Users\Administrator>cd c:\rc c:\rc>rclone authorize "onedrive" -·-·- （PowerShell 可能需要使用此命令：.\rclone authorize "onedrive"） Choose OneDrive account type? * Say b for a OneDrive business account * Say p for a personal OneDrive account b) Business p) Personal b/p> p If your browser doesn't open automatically go to the following link: http://127.0.0.1:53682/auth Log in and authorize rclone for access Waiting for code... Got code Paste the following into your remote machine ---> {"access_token":"***************########################################################################################################################**************************************************************************************************************************************************************************************************************$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$","expiry":"2018-09-01T01:43:55.5290909+08:00"} <---End paste 然后将 { } 内的内容（包括{}）复制到前面 result> 后面，回车即可。同样，执行 ls 或者 lsd 命令检查是否连接成功：#列出顶级文件夹，（gdrive 是前面第一步设置的名字，如果配置设置了密码会提示输入密码） rclone lsd onedrive: #列出所有文件 rclone ls onedrive: 2.1 定时执行脚本 rclone 命令大全如果用来备份或者同步文件，用到的命令可能就两个：rclone_copy 和 rclone_sync。rclone_copy 是复制，配合附加参数 --ignore-existing 可以假装增量备份；rclone_sync 是源目录同步到目标目录，使两者一致，会删除目标目录多余内容，测试时可以加上 --dry-run，检查会复制和删除那些文件。rclone about - Get quota information from the remote. rclone authorize - Remote authorization. rclone cachestats - Print cache stats for a remote rclone cat - Concatenates any files and sends them to stdout. rclone check - Checks the files in the source and destination match. rclone cleanup - Clean up the remote if possible rclone config - Enter an interactive configuration session. rclone copy - Copy files from source to dest, skipping already copied rclone copyto - Copy files from source to dest, skipping already copied rclone copyurl - Copy url content to dest. rclone cryptcheck - Cryptcheck checks the integrity of a crypted remote. rclone cryptdecode - Cryptdecode returns unencrypted file names. rclone dbhashsum - Produces a Dropbox hash file for all the objects in the path. rclone dedupe - Interactively find duplicate files and delete/rename them. rclone delete - Remove the contents of path. rclone deletefile - Remove a single file from remote. rclone genautocomplete - Output completion script for a given shell. rclone gendocs - Output markdown docs for rclone to the directory supplied. rclone hashsum - Produces an hashsum file for all the objects in the path. rclone link - Generate public link to file/folder. rclone listremotes - List all the remotes in the config file. rclone ls - List the objects in the path with size and path. rclone lsd - List all directories/containers/buckets in the path. rclone lsf - List directories and objects in remote:path formatted for parsing rclone lsjson - List directories and objects in the path in JSON format. rclone lsl - List the objects in path with modification time, size and path. rclone md5sum - Produces an md5sum file for all the objects in the path. rclone mkdir - Make the path if it doesn’t already exist. rclone mount - Mount the remote as a mountpoint. EXPERIMENTAL rclone move - Move files from source to dest. rclone moveto - Move file or directory from source to dest. rclone ncdu - Explore a remote with a text based user interface. rclone obscure - Obscure password for use in the rclone.conf rclone purge - Remove the path and all of its contents. rclone rc - Run a command against a running rclone. rclone rcat - Copies standard input to file on remote. rclone rmdir - Remove the path if empty. rclone rmdirs - Remove empty directories under the path. rclone serve - Serve a remote over a protocol. rclone sha1sum - Produces an sha1sum file for all the objects in the path. rclone size - Prints the total size and number of objects in remote:path. rclone sync - Make source and dest identical, modifying destination only. rclone touch - Create new file or change file modification time. rclone tree - List the contents of the remote in a tree like fashion. rclone version - Show the version number. 保证数据不会中断yum install screen screen -S rclone #第一次同步大量数据使用 screen 会话尽可能保护 rclone 执行不被中断 rclone copy gdrive:GooglePhotos onedrive:GooglePhotos 第一次同步完成，就可以 VPS 上（每天/周/月，按需选择）定时执行脚本自动同步。可以选择使用 rclone_copy --ignore-existing 进行增量备份（没有实测，按官方说明是可以实现），或者使用 rclone sync 同步。我希望 OneDrive 上的备份与 Google Drive 一致，所以每天定时执行 rclone sync gdrive:GooglePhotos onedrive:GooglePhotos/usr/bin/rclone sync gdrive:GooglePhotos onedrive:GooglePhotos >> /root/rclone.log 2>&1 配合 crontab 定时执行，比如每天 4:44 执行一次，日志写到 /root/rclone.log。执行 crontab -e，i，添加下面代码进去，按一下 Esc，然后输入 :wq 回车保存退出。44 4 * * * /usr/bin/rclone sync gdrive:GooglePhotos onedrive:GooglePhotos >> /root/rclone.log 2>&1 注：不需要日志的话把 >> /root/rclone.log 2>&1 去掉。然后重启一下 crond 服务即可。（service crond restart）注意：最好执行 service crond status 检查 crond 服务是否启动，service crond start 可手动启动，或者设置开机启动：chkconfig --level 35 crond on。Centos 7 x86_64 实测通过。 原文链接 https://eblog.ink/archives/219/ 2020-06-02T02:04:00+00:00 1.打开 笔记 点击打开2.点击左上角File ———— upload notebook————选择解压出来的额750G.ipynb备注：文件地址下载解压3.上传成功以后根据提示就可以了注意 第二次使用：1.打开链接2.找到Colab Notebooks文件夹打开3.右键750G.ipynb 打开方式选择 Colaboratory4.执行 第一步 第三步 和拉倒最下面一步 https://eblog.ink/archives/218/ 2020-05-19T13:23:00+00:00 1.获取PC版本Rclone 下载地址2.解压文件，cmd前往rclone的目录3.开始配置#打开菜单 rclone config #新增远程 n #输入名字（随便填） Goo #选择GoogleDrive 13 #client_id保持默认 回车 #client_secret保持默认 回车 #赋予全部权限 1 #root_folder_id保持默认 回车 #service_account_file保持默认 回车 #不编辑高级参数 n #使用自动config，并在弹出的网页登陆 y #是否添加为团队盘 n #确认参数正确 y 配置完成后输入 rclone config file 查看配置文件目录然后使用notepad++打开文件查看1/开头的 就是token了 https://eblog.ink/archives/217/ 2020-05-19T13:22:00+00:00 1.登陆cloudflare2.点击右边的Workers3.点击创建4.把index.js文件里面的所有内容都复制粘贴到左边脚本里面文件地址 密码:diqh5.网站名称、密码、Token就是上一篇文章获取到的1/开头的6.根目录ID为你go网盘的xxxxxxxxxxxxx例如： https://drive.google.com/drive/folders/xxxxxxxxxxxxx7.确定都正确以后点击保存并部署 https://eblog.ink/archives/216/ 2020-05-19T13:03:00+00:00 安装EPEL源yum -y install epel-release yum -y install wget unzip screen fuse fuse-devel wget https://downloads.rclone.org/v1.40/rclone-v1.40-linux-amd64.zip unzip rclone-v1.40-linux-amd64.zip cd rclone-v1.40-linux-amd64 ./rclone config 接下来就是选择e) Edit existing remote n) New remote d) Delete remote r) Rename remote c) Copy remote s) Set configuration password q) Quit config e/n/d/r/c/s/q> 选择nname> 随便输入 \ "google cloud storage" 11 / Google Drive \ "drive"Storage选择google 对应的数字client_id> 回车 client_secret> 回车 1 / Full access all files, excluding Application Data Folder. \ "drive" 2 / Read-only access to file metadata and file contents. \ "drive.readonly" / Access to files created by rclone only. 3 | These are visible in the drive website. | File authorization is revoked when the user deauthorizes the app. \ "drive.file" / Allows read and write access to the Application Data folder. 4 | This is not visible in the drive website. \ "drive.appfolder" / Allows read-only access to file metadata but 5 | does not allow any access to read or download file content. \ "drive.metadata.readonly" scope> 选择1root_folder_id>回车 service_account_file> 回车 Use auto config? * Say Y if not sure * Say N if you are working on a remote or headless machine or Y didn't work y) Yes n) No y/n>选择n If your browser doesn't open automatically go to the following link: https://accounts.google.com/o/xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx Log in and authorize rclone for access Enter verification code>这里输入复制上方的网址打开后获得的4/开头的码 configure this as a team drive? y)yes n)no y/n)如果出现这个选择n 完成后输入./rclone config file获取完成后输入vi ~/config/rclone/rclone.conf 获取到1/开头的就是token了