深度学习训练时,自动发送富文本邮件(含训练曲线图)给自己

前期准备

首先要有一个163邮箱(曾试过qq邮箱,但是效果不好),然后开通stmp业务。具体操作不赘述了,参见这篇博客:
https://blog.csdn.net/weixin_40475396/article/details/78693408

发邮件函数

参见:
https://www.cnblogs.com/zhongfengshan/p/9763366.html
这是我自己的函数,用我的163邮箱发给qq邮箱:

import smtplib
from email.mime.text import MIMEText

def sentemail(subject, body):
    ''' reference: https://www.cnblogs.com/zhongfengshan/p/9763366.html '''
    host = 'smtp.163.com'
    # 设置发件服务器地址
    port = 465
    # 设置发件服务器端口号。注意,这里有SSL和非SSL两种形式,现在一般是SSL方式
    sender = '[email protected]'
    # 设置发件邮箱,一定要自己注册的邮箱
    pwd = 'password'
    # 设置发件邮箱的授权码密码,根据163邮箱提示,登录第三方邮件客户端需要授权码
    receiver = '[email protected]'
    # 设置邮件接收人,可以是QQ邮箱
    # body = '<h1>标题</h1><p>内容</p>'
    # 设置邮件正文,这里是支持HTML的
    msg = MIMEText(body, 'html')
    # 设置正文为符合邮件格式的HTML内容
    msg['subject'] = subject
    # 设置邮件标题
    msg['from'] = sender
    # 设置发送人
    msg['to'] = receiver
    # 设置接收人
    try:
        s = smtplib.SMTP_SSL(host, port)
        # 注意!如果是使用SSL端口,这里就要改为SMTP_SSL
        s.login(sender, pwd)
        # 登陆邮箱
        s.sendmail(sender, receiver, msg.as_string())
        # 发送邮件!
        print('Done.sent email success, receiver is {}'.format(receiver))
    except smtplib.SMTPException:
        print('Error.sent email fail')

形成邮件正文(富文本)

邮件正文可以用html标签描述的富文本。我想实现把训练也发送过来,于是想到了用base64的方式。

绘制训练曲线,并保存为svg格式,返回保存路径

svg是一种矢量图像格式,适合于保存训练曲线。
我通过编写一下函数,进行lossaccuracy曲线的绘制。你也可以自己写。最后通过plt.savefig()来保存模型

import pylab as plt
from pathlib import Path

def draw(inputs, ylabel, title, ftype='svg'):
    ''' draw training figure. for example: >>>loss_img_path=draw([ >>>...[[(x1,y1),(x2,y2)...], 'train loss'], >>>...[[(x1,y1),(x2,y2)...], 'test loss'], >>>], ylabel='loss value', title=f'epoch0-loss') :param inputs: a list of curves. :param ylabel: figure ylabel :param title: figure title :param ftype: figure type, such as `png` `jpg` and so on. but `svg` is recommend :return: figure path '''
    figpath = Path(os.environ.get('SAVEDPATH', './')) / 'figure'
    figpath.mkdir(parents=True, exist_ok=True)
    for curve, name in inputs:
        X = []
        Y = []
        for x, y in curve:
            X.append(x)
            Y.append(y)
        plt.plot(X, Y, label=name)
    plt.title(title)
    plt.xlabel('train iterations')
    plt.ylabel(ylabel)
    plt.legend(loc='best')
    path = str(figpath / "{}.{}".format(title, ftype))
    plt.savefig(path)
    plt.close()
    return path

读取svg文件,用base64进行编码

import base64

def file2base64(fname):
    with open(fname, "rb") as f:
        base64_data = base64.b64encode(f.read())
    return base64_data

将base64编码的字符串用img标签包裹

def svgpath2htmlTag(svgpath):
    base = file2base64(svgpath)
    tag = "<img src='data:image/svg+xml;base64," + repr(base.decode())[1:] + "/>"
    return tag

大概的形式类似

<img src='data:image/svg+xml;base64,{svg文件的二进制编码转base64}'/>

当然你也可以传输其他的图片格式,例如png

形成正文

上面的内容主要讲得是对图片进行编码。其实正文的形成的难点也在此。解决了图片问题,问题就简单了。

from html import escape
from pathlib import Path

def generateBody(epoch, paths: list, logpath):
    with open(logpath) as f:
        log = f.read()
    figure_html = ''
    for path in paths:
        title = Path(path).stem
        figure_html += ''' <h2>{} figure</h2> {} <hr> '''.format(title, svgpath2htmlTag(path))

    body = ''' <h1>epoch-{} trainning report</h1> <hr> {} <h2>log details</h2> <pre> {} </pre> '''.format(epoch, figure_html, escape(log))
    return body

发送邮件后保存日志

from pathlib import Path

def generate_report(epoch, paths: list, logpath):
    body = generateBody(epoch, paths, logpath)
    sentemail('epoch-{}'.format(epoch), body)
    dname = Path(os.environ.get('SAVEDPATH', './')) / 'reports'
    dname.mkdir(exist_ok=True, parents=True)
    path = str(dname / 'report-{}.html'.format(epoch))
    with open(path, 'w+') as f:
        f.write(body)

效果展示

html展示

  • 以下为html直接粘贴在博客上(不要吐槽训练曲线)

epoch-4 trainning report


epoch4-loss figure


epoch4-acc figure


log details

/home/drug/drug_vqa/saved-dir/job-3554-saved/logs/epoch-4_06-06-11:51.log
epoch = 4
use prepared protein_dict
start test
use prepared protein_dict
test-actives :	 Test Loss: 0.125190,	 Acc: 0.970000	 Time: 0.88103s
use prepared protein_dict

手机截屏

在这里插入图片描述


更多精彩内容