Bioinformatics Tutorial
Files Needed
  • Getting Started
    • Setup
    • Run jobs in a Docker
    • Run jobs in a cluster [Advanced]
  • Part I. Programming Skills
    • 1.Linux
      • 1.1.Basic Command
      • 1.2.Practice Guide
      • 1.3.Linux Bash
    • 2.R
      • 2.1.R Basics
      • 2.2.Plot with R
    • 3.Python
  • PART II. BASIC ANALYSES
    • 1.Blast
    • 2.Conservation Analysis
    • 3.Function Analysis
      • 3.1.GO
      • 3.2.KEGG
      • 3.3.GSEA
    • 4.Clinical Analyses
      • 4.1.Survival Analysis
  • Part III. NGS DATA ANALYSES
    • 1.Mapping
      • 1.1 Genome Browser
      • 1.2 bedtools and samtools
    • 2.RNA-seq
      • 2.1.Expression Matrix
      • 2.2.Differential Expression with Cufflinks
      • 2.3.Differential Expression with DEseq2 and edgeR
    • 3.ChIP-seq
    • 4.Motif
      • 4.1.Sequence Motif
      • 4.2.Structure Motif
    • 5.RNA Network
      • 5.1.Co-expression Network
      • 5.2.miRNA Targets
      • 5.3. CLIP-seq (RNA-Protein Interaction)
    • 6.RNA Regulation - I
      • 6.1.Alternative Splicing
      • 6.2.APA (Alternative Polyadenylation)
      • 6.3.Chimeric RNA
      • 6.4.RNA Editing
      • 6.5.SNV/INDEL
    • 7.RNA Regulation - II
      • 7.1.Translation: Ribo-seq
      • 7.2.RNA Structure
    • 8.cfDNA
      • 8.1.Basic cfDNA-seq Analyses
  • Part IV. MACHINE LEARNING
    • 1.Machine Learning Basics
      • 1.1 Data Pre-processing
      • 1.2 Data Visualization & Dimension Reduction
      • 1.3 Feature Extraction and Selection
      • 1.4 Machine Learning Classifiers/Models
      • 1.5 Performance Evaluation
    • 2.Machine Learning with R
    • 3.Machine Learning with Python
  • Part V. Assignments
    • 1.Precision Medicine - exSEEK
      • Help
      • Archive: Version 2018
        • 1.1.Data Introduction
        • 1.2.Requirement
        • 1.3.Helps
    • 2.RNA Regulation - RiboShape
      • 2.0.Programming Tools
      • 2.1.RNA-seq Analysis
      • 2.2.Ribo-seq Analysis
      • 2.3.SHAPE Data Analysis
      • 2.4.Integration
    • 3.RNA Regulation - dsRNA
    • 4.Single Cell Data Analysis
      • Help
  • 5.Model Programming
  • Appendix
    • Appendix I. Keep Learning
    • Appendix II. Databases & Servers
    • Appendix III. How to Backup
    • Appendix IV. Teaching Materials
    • Appendix V. Software and Tools
    • Appendix VI. Genome Annotations
Powered by GitBook
On this page
  • 1) Basic
  • 2) Advanced
  • (1) Backup your code with GitHub in Terminal
  • (2) Backup data using rsync and crontab
  • (3) More Reading for advanced users

Was this helpful?

Edit on GitHub
  1. Appendix

Appendix III. How to Backup

PreviousAppendix II. Databases & ServersNextAppendix IV. Teaching Materials

Last updated 2 years ago

Was this helpful?

1) Basic

  • see in Getting Started.

2) Advanced

(1) Backup your code with GitHub in Terminal

  • Setup

    • (optinal)

    • add a setting file: ~/.gitconfig

[user] 
email =[NNN@NN.com]
name = Shared
  • Clone/Download an existed repository on github

git clone git@github.com:lulab/RNAfinder_Server.git
git clone https://github.com/xug15/test.git
  • Create a new repository

echo "# test" >> README.md 
git init 
git add README.md 
git commit -m "first commit" 
git remote add origin https://github.com/xug15/test.git 
git push -u origin master
  • Sync local files with github repo

Pull (update):

git pull origin master
git log -n 2 # look at the last two log entries.

Add:

git add exmaples/
git commit -m ‘20190705v1’
git push origin

Change:

git commit -m ‘20190705v1’
git push origin

Remove:

git rm *.file
git commit -m ‘20190705v1’
git push origin

Tips: the bash script to sync a github repo:

time=`date`
echo $time
git add -u .
git add *
git commit -m '$time'
git push origin master

(2) Backup data using rsync and crontab

(2.1) Setup ssh key (optional)

  • (a) Generate SSH key

ssh-keygen -t rsa -b 2048
  • (b) Copy your keys to the target server

ssh-copy-id user@server_ip    #if port add: -p 2200

上述操作后,通过ssh到remote server时就可以无需输入密码了,因此下面的步骤也可以在无人值守的时候自动运行。

但如果下面的步骤中你无需登录remote server, 就无需setup ssh key。

(2.2) Prepare a backup script with rsync

  • (a) First you need to prepare some backup dirs

mkdir /home/john/backup_local    # prepare a backup dir for some local files
mkdir /home/john/backup_remote   # prepare a backup dir for some remote files
  • (b) Then, write a back up script, for example : ~/backup.sh

#!/bin/bash

#0. Define the parameters of rsync
RSYNC="rsync --stats  --compress --recursive --times --perms --links --delete --max-size=100M --exclude-from=/home/john/excluded_file_list.txt"

#A. Local backup  
echo "1. Backup of /home/john/data start at:"
date
$RSYNC /home/john/data/  /home/john/backup_local/
echo "Backup end at:"
date

#B. Remote backup 
echo "2. Backup 166.178.56.20:/home/lulab/john/data/ start at:"
date
$RSYNC john@166.178.56.20:/home/lulab/john/data/ /home/john/backup_remote/
echo "Backup end at:"
date
  • (c) Last, make your backup.sh excutable

chmod +x ~/backup.sh

Parameters of rsync (use man rsync to see more details):

Parameter

Mean

-a:

以递归方式传输文件

--delete:

删除那些接收端还有而发送端已经不存在的文件

-q:

精简输出模式

-z:

在传输文件时进行压缩处理

-H:

保持硬链接文件

-t:

对比两边文件的时间戳和文件大小.如果一致,则就认为两边文件一样,对此文件就不再采取更新动作了

-I:

挨个文件去发起数据同步

--port=PORT:

端口号

(2.3) Schedule the back tasks with crontab

  • 打开crontab编辑器:

crontab -e
  • 加入以下行:

# minute hour day_in_month month day_in_week command
15 3 * * * /home/john/backup.sh > /home/john/backup.log

Linux将通过crontab定时运行上述命令, 具体定义如下:

Column

Mean

Column 1:

Minutes 0 to 59

Column 2:

Hours 0 to 23 (0 means midnight)

Column 3:

Day 1 to 31

Column 4:

Months 1~12

Column 5:

Week 0 to 7 (0 and 7 for Sunday)

Column 6:

Command to run

(3) More Reading for advanced users

  • Linux 推荐章节:

    • 第25章 LINUX备份策略: 25.2.2完整备份的差异备份; 25.3鸟哥的备份策略; 25.4灾难恢复的考虑; 25.5重点回顾

crontab是Linux中用来定期执行程序的命令, 你可以使用 ,也可以按如下方式自己编辑:

《》 (25章推荐章节)

在线crontab生成器
鸟哥的Linux私房菜-基础学习篇
set up ssh-key
Backup your work - Basic