GeorgeYang'Blog

my technology blog

python-webpy和mongodb实现博客统计

阅读:382 创建时间:16-02-13 10:18:43 tags:python,mongo

本宝宝从0开始写的博客一直没有博客统计功能,于是计划加上统计功能!!

step:0x001

python使用pymongo登录远程数据库 mongodb默认本地就可以直接访问,设置密码后要这样才能访问

         client = MongoClient('host','port')
         ret = client['admin'].authenticate('username', 'password')
         print ret
     return client['dbname']

step:0x010

表设计

 #今日ip访问表
 # connectStatistics   agent day(like:2015-11-11) ip blogId(if not blogId = -1)

 #博客uv,pv历史表
 # blogStatistics   day(like:2015-11-11) pv uv

统计思路及代码

先获取connectStatistics表最后插入一条的记录,获取时间(格式:yyyy-mm-dd),用该时间与系统时间对比,如果不一致,则是昨天的记录,此时统计connectStatistics的pv,uv数据,将输入插入blogStatistics历史访问表,成功后删除访问表的全部数据,开始新一天的博客数据收集统计。

 saveDate = time.strftime("%y-%m-%d",time.localtime())

         oneRecd =  ut.getMongoDb().connectStatistics.find_one()

         if oneRecd:
             lastSaveDate = oneRecd['day']

             if not saveDate == lastSaveDate:
                 print 'del old data,the date is:%s'%lastSaveDate

                 blogStat = {}
                 blogStat['day'] = lastSaveDate
                 blogStat['pv'] = but.getPvToday()
                 blogStat['uv'] = but.getUvToday()

                 mongodb.blogStatistics.insert(blogStat)
                 mongodb.connectStatistics.remove({"day":lastSaveDate})

step:0x011

收集今日记录:

在统一访问入口开启ip访问记录线程:

 import thread


         agent = web.ctx.env['HTTP_USER_AGENT']
         remoteAddr = web.ctx.env['REMOTE_ADDR']
         params = ut.getInput(web.input())
         if params.has_key('id'):
             blogId = params['id']
         else:
             blogId = '-1'
 thread.start_new_thread (self.saveIp,(blogId, agent,remoteAddr))


 def saveIp (self,blogId,agent,remoteAddr):

         if agent==None or len(agent)==0:
             return

         #文章记录 readCount
         blog = mongodb.find_one({'id':blogId})
         print "blog:"
         print blog
         if blog:
             #博客阅读数据+1
             mongodb.blog.update({"id": blogId}, {"$set": {"readCount": blog['readCount']+1}})
         #end

         saveDate = time.strftime("%y-%m-%d",time.localtime())

         oneRecd =  mongodb.connectStatistics.find_one()

         if oneRecd:
             lastSaveDate = oneRecd['day']

             if not saveDate == lastSaveDate:
                 print 'del old data,the date is:%s'%lastSaveDate

                 blogStat = {}
                 blogStat['day'] = lastSaveDate
                 # blogStat['pv'] = mongodb.connectStatistics.count()
                 blogStat['pv'] = but.getPvToday()
                 blogStat['uv'] = but.getUvToday()

                 mongodb.blogStatistics.insert(blogStat)
                 mongodb.connectStatistics.remove({"day":lastSaveDate})

         data = {}
         data['blogId'] = blogId
         data['agent']=agent
         data['ip']=remoteAddr
         data['day']= saveDate

         mongodb.connectStatistics.insert(data)

 def getPvToday():
     try:
         return mongodb.connectStatistics.count()
     except Exception, e:
         print e
     return 0



 def getUvToday():
     try:
         key = ['ip']
         cond = None
         initial = {'count' : 0}
         reduce = "function (obj, prev) { prev.count++; }"
         #按ip分组,计算组数
         group = list(mongodb.connectStatistics.group(key,cond,initial,reduce))
         uv = len(group)
         return uv
     except Exception, e:
         print e
     return 0

step:0x100

显示在html

新建工具类返回blog访问记录map:

 #获取博客统计
 def getWebInfo():
     cacheKey = 'blogutil-getwebinfo-key'
     ret = lrucache.getinstance().get(cacheKey)
     if ret:
         print 'use cache:' + cacheKey
         return ret

     info = {}
     info['uv'] = getUvToday()
     info['pv'] = getPvToday()
     info['readCount'] = int(getBlogPv())

     #两分钟的缓存
     lrucache.getinstance().set(cacheKey,info,120)

     return info

代码获取访问数据,传给templates显示

 return ut.getTemplatesWithBase().showcontent(util.getWebInfo(),blogInfo,newLast,monthList,classificationMap)

显示数据

 <div class="lm" id="menu_card">
                        <ul>
                            网站统计:
                            <li>
                                总访问量:$:webInfo['readCount']
                            </li>

                            <li>
                                今日uv:$:webInfo['uv']
                            </li>

                            <li>
                                今日pv:$:webInfo['pv']
                            </li>
                        </ul>
                    </div>