問題描述

我嘗試每天從我的服務(wù)器下載一個備份文件到我的本地存儲服務(wù)器，但我遇到了一些問題.

Im trying to download daily a backup file from my server to my local storage server, but i got some problems.

我寫了這段代碼(去掉了無用的部分，作為電子郵件功能):

I wrote this code (removed the useless parts, as the email function):

import os
from time import strftime
from ftplib import FTP
import smtplib
from email.MIMEMultipart import MIMEMultipart
from email.MIMEBase import MIMEBase
from email.MIMEText import MIMEText
from email import Encoders

day = strftime("%d")
today = strftime("%d-%m-%Y")

link = FTP(ftphost)
link.login(passwd = ftp_pass, user = ftp_user)
link.cwd(file_path)
link.retrbinary('RETR ' + file_name, open('/var/backups/backup-%s.tgz' % today, 'wb').write)
link.delete(file_name) #delete the file from online server
link.close()
mail(user_mail, "Download database %s" % today, "Database sucessfully downloaded: %s" % file_name)
exit()

我使用 crontab 運行它，例如:

And i run this with a crontab like:

40    23    *    *    *    python /usr/bin/backup-transfer.py >> /var/log/backup-transfer.log 2>&1

它適用于小文件，但它會凍結(jié)備份文件(大約 1.7Gb)，下載的文件大約 1.2Gb 然后永遠不會增長(我等了大約一天)，并且日志文件是空的.

It works with small files, but with the backups files (about 1.7Gb) it freeze, the downloaded file get about 1.2Gb then never grows up (i waited about a day), and the log file is empty.

有什么想法嗎?

ps:我使用的是 Python 2.6.5

p.s: im using Python 2.6.5

推薦答案

對不起，如果我回答了我自己的問題，但我找到了解決方案.

Sorry if i answer my own question, but I found the solution.

我嘗試了 ftputil 沒有成功，所以我嘗試了很多方法，最后，這行得通:

I tryed ftputil with no success, so i tryed many way and finally, this works:

def ftp_connect(path):
    link = FTP(host = 'example.com', timeout = 5) #Keep low timeout
    link.login(passwd = 'ftppass', user = 'ftpuser')
    debug("%s - Connected to FTP" % strftime("%d-%m-%Y %H.%M"))
    link.cwd(path)
    return link

downloaded = open('/local/path/to/file.tgz', 'wb')

def debug(txt):
    print txt

link = ftp_connect(path)
file_size = link.size(filename)

max_attempts = 5 #I dont want death loops.

while file_size != downloaded.tell():
    try:
        debug("%s while > try, run retrbinary
" % strftime("%d-%m-%Y %H.%M"))
        if downloaded.tell() != 0:
            link.retrbinary('RETR ' + filename, downloaded.write, downloaded.tell())
        else:
            link.retrbinary('RETR ' + filename, downloaded.write)
    except Exception as myerror:
        if max_attempts != 0:
            debug("%s while > except, something going wrong: %s
 	file lenght is: %i > %i
" %
                (strftime("%d-%m-%Y %H.%M"), myerror, file_size, downloaded.tell())
            )
            link = ftp_connect(path)
            max_attempts -= 1
        else:
            break
debug("Done with file, attempt to download m5dsum")
[...]

在我的日志文件中我發(fā)現(xiàn):

In my log file i found:

01-12-2011 23.30 - Connected to FTP
01-12-2011 23.30 while > try, run retrbinary
02-12-2011 00.31 while > except, something going wrong: timed out
    file lenght is: 1754695793 > 1754695793
02-12-2011 00.31 - Connected to FTP
Done with file, attempt to download m5dsum

遺憾的是，即使文件已完全下載，我也必須重新連接到 FTP，這在我的 cas 中不是問題，因為我也必須下載 md5sum.

Sadly, i have to reconnect to FTP even if the file has been fully downloaded, that in my cas is not a problem, becose i have to download the md5sum too.

如您所見，我無法檢測到超時并重試連接，但是當(dāng)我超時時，我只是重新連接；如果有人知道如何在不創(chuàng)建新的 ftplib.FTP 實例的情況下重新連接，請告訴我 ;)

As you can see, I'm not been able to detect the timeout and retry the connection, but when i got timeout, I simply reconnect again; If someone know how to reconnect without creating a new ftplib.FTP instance, let me know ;)

這篇關(guān)于使用python通過FTP下載大文件的文章就介紹到這了，希望我們推薦的答案對大家有所幫助，也希望大家多多支持html5模板網(wǎng)！