python 实现过滤出tomcat日志中含有ERROR 或Exception 的行并保存在另一个文件

遍历多个tomcat日志文件，找出含有ERROR 和Exception 的日志，并把该行日志输出到另一个文件中：（这里为了体现python模块导入的知识，所有建立了多个文件夹和模块）

项目结构：

consetting.py:

# 日志文件目录
F_PATH = r'C:\Users\shenping\PycharmProjects\Shenping_TEST\day_5\script\glive\logs'
# 错误日志存储目录
D_PATH = r'C:\Users\shenping\PycharmProjects\Shenping_TEST\day_5\script\glive\data'

rwfile.py:

def op_file(filename,content =None):
    f = open(filename,'a+',encoding='utf-8')
    f.seek(0)
    if content:
        f.writelines(content)
        res = None
    else:
        res = f.readlines()
    f.close()
    return res

operationlog.py:

import re,os,datetime
from lib.rwfile import op_file

# 遍历日志文件：
# 找出含有 error 或 exception 的行，按格式写入文件
def operation_log(filename,f_path,d_path):
    index = 0
    log_file = os.path.join(f_path, filename)
    f = op_file(log_file)
    lis = []  # 存储错误日志
    for j in f:
        index += 1
        m = re.search('error',j,re.IGNORECASE)
        n = re.search('exception',j,re.IGNORECASE)
        if m or n:
            data_file = os.path.join(d_path, str(datetime.date.today())+'_tomcat.log')
            log_str = "文件名："+filename+" 第"+str(index)+"行："+j+'\n'
            lis.append(log_str)
    op_file(data_file,lis)

 start.py:

# 启动文件
import os,sys

cur_path = os.path.abspath(__file__)
base_dir = os.path.dirname(os.path.dirname(cur_path))
sys.path.insert(0,base_dir)
from conf.consetting import F_PATH,D_PATH
from lib.operationlog import operation_log

log_name = os.listdir(F_PATH)
for x in log_name:
    operation_log(x,F_PATH,D_PATH)
    print('========================================华丽的分割线=========================================================')

运行后结果：

文件名：catalina.2017-10-21.log 第5行：21-Oct-2017 14:16:32.118 ERROR [localhost-startStop-2] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [ROOT] appears to have started a thread named [Abandoned connection cleanup thread] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
文件名：catalina.2017-10-21.log 第9行：21-Oct-2017 14:16:32.119 ERROR [localhost-startStop-2] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [ROOT] appears to have started a thread named [threadDeathWatcher-2-1] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
文件名：catalina.2017-10-21.log 第14行：21-Oct-2017 14:16:32.120 ERROR [localhost-startStop-2] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [ROOT] appears to have started a thread named [logback-1] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
文件名：catalina.2017-10-21.log 第515行：21-Oct-2017 16:08:39.217 ERROR [localhost-startStop-2] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [ROOT] appears to have started a thread named [glive-task-pool-1-thread-12] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
文件名：catalina.2017-10-22.log 第216行：22-Oct-2017 15:17:49.169 Exception [localhost-startStop-2] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [ROOT] appears to have started a thread named [Abandoned connection cleanup thread] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
文件名：catalina.2017-10-22.log 第248行：22-Oct-2017 15:17:54.089 Exception [main] org.apache.tomcat.util.digester.SetPropertiesRule.begin [SetPropertiesRule]{Server/Service/Engine/Host/Context} Setting property 'debug' to '' did not find a matching property.
文件名：catalina.2017-10-22.log 第249行：22-Oct-2017 15:17:54.113 Exception [main] org.apache.tomcat.util.digester.SetPropertiesRule.begin [SetPropertiesRule]{Server/Service/Engine/Host/Valve} Setting property 'resolveHosts' to 'false' did not find a matching property.
文件名：catalina.2017-10-23.log 第3行：23-Oct-2017 10:51:55.461 IllegalStateException [localhost-startStop-2] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesJdbc The web application [ROOT] registered the JDBC driver [com.mysql.jdbc.Driver] but failed to unregister it when the web application was stopped. To prevent a memory leak, the JDBC Driver has been forcibly unregistered.
文件名：catalina.2017-10-23.log 第4行：23-Oct-2017 10:51:55.462 ERROR [localhost-startStop-2] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesJdbc The web application [ROOT] registered the JDBC driver [com.mysql.fabric.jdbc.FabricMySQLDriver] but failed to unregister it when the web application was stopped. To prevent a memory leak, the JDBC Driver has been forcibly unregistered.
文件名：catalina.2017-10-23.log 第9行：23-Oct-2017 10:51:55.464 IllegalStateException [localhost-startStop-2] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [ROOT] appears to have started a thread named [threadDeathWatcher-3-1] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
文件名：catalina.2017-10-23.log 第3356行：23-Oct-2017 21:47:50.795 IllegalStateException [logback-1] org.apache.catalina.loader.WebappClassLoaderBase.checkStateForResourceLoading Illegal access: this web application instance has been stopped already. Could not load [ch.qos.logback.core.spi.ContextAwareBaseBeanINFO]. The following stack trace is thrown for debugging purposes as well as to attempt to terminate the thread which caused the illegal access.
文件名：catalina.2017-10-23.log 第3357行： java.lang.IllegalStateException: Illegal access: this web application instance has been stopped already. Could not load [ch.qos.logback.core.spi.ContextAwareBaseBeanINFO]. The following stack trace is thrown for debugging purposes as well as to attempt to terminate the thread which caused the illegal access.
文件名：catalina.2017-10-23.log 第3398行：23-Oct-2017 21:47:50.800 ERROR [logback-1] org.apache.catalina.loader.WebappClassLoaderBase.checkStateForResourceLoading Illegal access: this web application instance has been stopped already. Could not load [java.lang.ObjectBeanINFO]. The following stack trace is thrown for debugging purposes as well as to attempt to terminate the thread which caused the illegal access.
文件名：catalina.2017-10-23.log 第3399行： java.lang.IllegalStateException: Illegal access: this web application instance has been stopped already. Could not load [java.lang.ObjectBeanINFO]. The following stack trace is thrown for debugging purposes as well as to attempt to terminate the thread which caused the illegal access.
文件名：catalina.2017-10-23.log 第3444行：23-Oct-2017 21:47:50.802 ERROR [logback-1] org.apache.catalina.loader.WebappClassLoaderBase.checkStateForResourceLoading Illegal access: this web application instance has been stopped already. Could not load [java.lang.ObjectCustomizer]. The following stack trace is thrown for debugging purposes as well as to attempt to terminate the thread which caused the illegal access.
文件名：catalina.2017-10-23.log 第3445行： java.lang.IllegalStateException: Illegal access: this web application instance has been stopped already. Could not load [java.lang.ObjectCustomizer]. The following stack trace is thrown for debugging purposes as well as to attempt to terminate the thread which caused the illegal access.
文件名：catalina.2017-10-23.log 第3491行： java.lang.IllegalStateException: Illegal access: this web application instance has been stopped already. Could not load [ch.qos.logback.core.spi.ContextAwareBaseCustomizer]. The following stack trace is thrown for debugging purposes as well as to attempt to terminate the thread which caused the illegal access.
文件名：catalina.2017-10-23.log 第3532行： java.lang.IllegalStateException: Illegal access: this web application instance has been stopped already. Could not load [ch.qos.logback.core.rolling.RollingPolicyBaseCustomizer]. The following stack trace is thrown for debugging purposes as well as to attempt to terminate the thread which caused the illegal access.
文件名：catalina.2017-10-23.log 第3570行： java.lang.IllegalStateException: Illegal access: this web application instance has been stopped already. Could not load [ch.qos.logback.core.rolling.TimeBasedRollingPolicyCustomizer]. The following stack trace is thrown for debugging purposes as well as to attempt to terminate the thread which caused the illegal access.
文件名：catalina.2017-10-23.log 第3605行： java.lang.IllegalStateException: Illegal access: this web application instance has been stopped already. Could not load [ch.qos.logback.core.rolling.helper.DateTokenConverter]. The following stack trace is thrown for debugging purposes as well as to attempt to terminate the thread which caused the illegal access.
文件名：catalina.2017-10-23.log 第3637行： java.lang.IllegalStateException: Illegal access: this web application instance has been stopped already. Could not load [ch.qos.logback.core.status.INFOStatus]. The following stack trace is thrown for debugging purposes as well as to attempt to terminate the thread which caused the illegal access.
文件名：catalina.2017-10-23.log 第3667行： java.lang.IllegalStateException: Illegal access: this web application instance has been stopped already. Could not load [ch.qos.logback.core.status.INFOStatus]. The following stack trace is thrown for debugging purposes as well as to attempt to terminate the thread which caused the illegal access.
文件名：catalina.2017-10-24.log 第5行：24-Oct-2017 08:02:51.428 IllegalStateException [localhost-startStop-2] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [ROOT] appears to have started a thread named [Abandoned connection cleanup thread] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
文件名：catalina.2017-10-24.log 第520行：24-Oct-2017 14:42:09.350 Exception [localhost-startStop-2] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [ROOT] appears to have started a thread named [Abandoned connection cleanup thread] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
文件名：catalina.2017-10-24.log 第670行：24-Oct-2017 14:42:09.374 Exception [localhost-startStop-2] org.apache.catalina.loader.WebappClassLoaderBase.checkThreadLocalMapForLeaks The web application [ROOT] created a ThreadLocal with key of type [java.lang.ThreadLocal] (value [java.lang.ThreadLocal@e5d976d]) and a value of type [io.netty.util.internal.InternalThreadLocalMap] (value [io.netty.util.internal.InternalThreadLocalMap@b0bd2a4]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
文件名：catalina.2017-10-24.log 第682行：24-Oct-2017 14:42:09.376 Exception [localhost-startStop-2] org.apache.catalina.loader.WebappClassLoaderBase.checkThreadLocalMapForLeaks The web application [ROOT] created a ThreadLocal with key of type [java.lang.ThreadLocal] (value [java.lang.ThreadLocal@55c7591]) and a value of type [io.grpc.Context] (value [io.grpc.Context@390c5a57]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
文件名：catalina.2017-10-24.log 第704行：24-Oct-2017 14:42:09.378 Exception [localhost-startStop-2] org.apache.catalina.loader.WebappClassLoaderBase.checkThreadLocalMapForLeaks The web application [ROOT] created a ThreadLocal with key of type [java.lang.ThreadLocal] (value [java.lang.ThreadLocal@55c7591]) and a value of type [io.grpc.Context] (value [io.grpc.Context@390c5a57]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.