Initial import of mwc version 1.7.2-1

author: Jörg Frings-Fürst <debian@jff-webhosting.net> 2014-09-19 08:49:38 +0200
committer: Jörg Frings-Fürst <debian@jff-webhosting.net> 2014-09-19 08:49:38 +0200
commit: e8136364adf237f9050b7c150c230abd38fff0c0 (patch)
tree: ac80b0b6a998cc1d7c3af2440588a9c113deb68b
parent: b329a335ad10648e64b3de049c083bc04638a072 (diff)
24 files changed, 468 insertions, 0 deletions
diff --git a/.gitignore b/.gitignore
index 4ccd411..4579214 100644
--- a/.gitignore
+++ b/.gitignore
@@ -4,4 +4,6 @@
 /*.txt
 *~
 *.pyc
+.bzrignore
+.bzr
 
diff --git a/debian/README.Debian b/debian/README.Debian
new file mode 100644
index 0000000..ebbea19
--- /dev/null
+++ b/debian/README.Debian
@@ -0,0 +1,18 @@
+mwc for Debian
+--------------
+
+1.) For a example based setup copy the example to /etc/mwc:
+
+    cp -f /usr/share/doc/mwc/examples/mwc-config_template.py /etc/mwc/mwc-config.py
+
+
+2.) For running every hour you can copy 
+
+    cp -f /usr/share/doc/mwc/examples/mwc /etc/cron.hourly/mwc
+
+3.) You find the documentation in the file README.md or on
+
+    https://github.com/Debianguru/MailWebsiteChanges#mailwebsitechanges 
+
+
+ -- Jörg Frings-Fürst <debian@jff-webhosting.net>  Thu, 18 Sep 2014 22:49:00 +0200
diff --git a/debian/README.source b/debian/README.source
new file mode 100644
index 0000000..066a444
--- /dev/null
+++ b/debian/README.source
@@ -0,0 +1,8 @@
+mwc for Debian
+--------------
+
+The documentation is a separate source tarball.
+It was copied to debian/doc/html.
+
+ -- Jörg Frings-Fürst <debian@jff-webhosting.net>  Sat, 03 May 2014 14:42:36 +0200
+
diff --git a/debian/changelog b/debian/changelog
new file mode 100644
index 0000000..90fdd4f
--- /dev/null
+++ b/debian/changelog
@@ -0,0 +1,5 @@
+mwc (1.7.2-1) unstable; urgency=low
+
+  * Initial release (Closes: #746556)
+
+ -- Jörg Frings-Fürst <debian@jff-webhosting.net>  Thu, 18 Sep 2014 10:49:00 +0200
diff --git a/debian/compat b/debian/compat
new file mode 100644
index 0000000..ec63514
--- /dev/null
+++ b/debian/compat
@@ -0,0 +1 @@
+9
diff --git a/debian/control b/debian/control
new file mode 100644
index 0000000..059e6d5
--- /dev/null
+++ b/debian/control
@@ -0,0 +1,29 @@
+Source: mwc
+Section: utils
+Priority: optional
+Maintainer: Jörg Frings-Fürst <debian@jff-webhosting.net>
+Build-Depends: 
+ debhelper (>= 9),
+ dh-python,
+ python3 | python3-all | python3-dev | python3-all-dev
+Standards-Version: 3.9.5
+Homepage: https://github.com/Debianguru/MailWebsiteChanges
+Vcs-Git: git://anonscm.debian.org/collab-maint/mwc.git
+Vcs-Browser: http://anonscm.debian.org/cgit/collab-maint/mwc.git
+
+Package: mwc
+Architecture: all
+Depends: 
+ ${misc:Depends},
+ ${python3:Depends},
+ python3 | python3-all | python3-dev | python3-all-dev,
+ python3-lxml,
+ python-cssselect
+Description: Powerful website-tracking tool
+ MailWebsiteChanges is a Python script to keep track of website
+ changes; sends email notifications on updates and/or also
+ provides an RSS feed.
+ .
+ To specify which parts of a website should be monitored, XPath
+ selectors, CSS selectors, and regular  expressions can be used
+ (just choose the tools you like!).
diff --git a/debian/copyright b/debian/copyright
new file mode 100644
index 0000000..c24e615
--- /dev/null
+++ b/debian/copyright
@@ -0,0 +1,45 @@
+Format: http://www.debian.org/doc/packaging-manuals/copyright-format/1.0/
+Upstream-Name: mwc
+Source: https://github.com/Debianguru/MailWebsiteChanges
+
+Files: *
+Copyright: (2013-2014) Michael Till Beck <Debianguru@gmx.de>
+License: GPL-2+
+
+Files: debian/*
+Copyright: 2014 Jörg Frings-Fürst <debian@jff-webhosting.net>
+License: GPL-3+
+
+License: GPL-2+
+ This package is free software; you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation; either version 2 of the License, or
+ (at your option) any later version.
+ .
+ This package is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ GNU General Public License for more details.
+ .
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>
+ .
+ On Debian systems, the complete text of the GNU General
+ Public License version 2 can be found in "/usr/share/common-licenses/GPL-2".
+
+License: GPL-3+
+ This package is free software; you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation; either version 3 of the License, or
+ (at your option) any later version.
+ .
+ This package is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ GNU General Public License for more details.
+ .
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>
+ .
+ On Debian systems, the complete text of the GNU General
+ Public License version 3 can be found in "/usr/share/common-licenses/GPL-3".
diff --git a/debian/dirs b/debian/dirs
new file mode 100644
index 0000000..870600a
--- /dev/null
+++ b/debian/dirs
@@ -0,0 +1,2 @@
+etc/mwc
+var/cache/mwc
diff --git a/debian/docs b/debian/docs
new file mode 100644
index 0000000..648393d
--- /dev/null
+++ b/debian/docs
@@ -0,0 +1,2 @@
+README.md
+
diff --git a/debian/mwc.1 b/debian/mwc.1
new file mode 100644
index 0000000..683b71b
--- /dev/null
+++ b/debian/mwc.1
@@ -0,0 +1,59 @@
+.\"                                      Hey, EMACS: -*- nroff -*-
+.\" (C) Copyright 2014 Jörg Frings-Fürst <debian@jff-webhosting.net>,
+.\"
+.\" First parameter, NAME, should be all caps
+.\" Second parameter, SECTION, should be 1-8, maybe w/ subsection
+.\" other parameters are allowed: see man(7), man(1)
+.TH MWC 1 "September  18, 2014"
+.\" Please adjust this date whenever revising the manpage.
+.\"
+.\" Some roff macros, for reference:
+.\" .nh        disable hyphenation
+.\" .hy        enable hyphenation
+.\" .ad l      left justify
+.\" .ad b      justify to both left and right margins
+.\" .nf        disable filling
+.\" .fi        enable filling
+.\" .br        insert line break
+.\" .sp <n>    insert n+1 empty lines
+.\" for manpage-specific macros, see man(7)
+.SH NAME
+mwc, mwcfeedserver \- program for monitoring changes of Websites
+.SH SYNOPSIS
+.B mwc
+.RI [ --config[=config]\ \ --dry-run[=shortname] ] 
+.br
+.B mwcfeedserver
+.RI [ --config[=config]\ \ --port[=port] ] 
+.SH DESCRIPTION
+This manual page documents briefly the
+.B mwc and
+.br
+.B mwcfeedserver
+commands.
+.PP
+.\" TeX users may be more comfortable with the \fB<whatever>\fP and
+.\" \fI<whatever>\fP escape sequences to invode bold face and italics,
+.\" respectively.
+\fBmwc\fP is a program that monitor websites and send mails after detect changes.
+.SH OPTIONS
+These programs follow the usual GNU command line syntax, with long
+options starting with two dashes (`-').
+A summary of options is included below.
+.TP
+.B \-\-config=CONFIG
+Path and filename of the config \- file.
+.TP
+.B \-\-dry\-run=SHORTNAME
+Test the entry SHORTNAME from config without sending a mail.
+.TP
+.B \-\-port=PORT
+Port (Standard = 8000) where Feedserver is listen
+.SH AUTHOR
+The Sleuth Kit was written by Michael Till Beck <Debianguru@gmx.de>.
+.PP
+This manual page was written by Jörg Frings\-Fürst
+.br
+<debian@jff\-webhosting.net> for the Debian project
+.br
+(but may be used by others).
diff --git a/debian/mwc.cr b/debian/mwc.cr
new file mode 100755
index 0000000..e0c7f8f
--- /dev/null
+++ b/debian/mwc.cr
@@ -0,0 +1,7 @@
+#! /bin/bash
+
+test -x /usr/bin/mwc || exit 0
+test -e /etc/mwc/mwc-config.py || exit 0
+
+/usr/bin/mwc -config=/etc/mwc/mwc-config  >/dev/null 2>&1
+
diff --git a/debian/mwc.examples b/debian/mwc.examples
new file mode 100644
index 0000000..9c2cf1d
--- /dev/null
+++ b/debian/mwc.examples
@@ -0,0 +1,2 @@
+config_template.py
+debian/mwc.cr
diff --git a/debian/mwc.install b/debian/mwc.install
new file mode 100644
index 0000000..d893da8
--- /dev/null
+++ b/debian/mwc.install
@@ -0,0 +1,2 @@
+mwc.py /usr/share/mwc
+mwcfeedserver.py /usr/share/mwc
diff --git a/debian/mwc.links b/debian/mwc.links
new file mode 100644
index 0000000..82885c9
--- /dev/null
+++ b/debian/mwc.links
@@ -0,0 +1,3 @@
+/usr/share/mwc/mwc.py /usr/bin/mwc
+/usr/share/mwc/mwcfeedserver.py /usr/sbin/mwcfeedserver
+
diff --git a/debian/mwc.manpages b/debian/mwc.manpages
new file mode 100644
index 0000000..c3ed826
--- /dev/null
+++ b/debian/mwc.manpages
@@ -0,0 +1,2 @@
+debian/mwc.1
+debian/mwcfeedserver.1
diff --git a/debian/mwcfeedserver.1 b/debian/mwcfeedserver.1
new file mode 120000
index 0000000..1cd4511
--- /dev/null
+++ b/debian/mwcfeedserver.1
@@ -0,0 +1 @@
+mwc.1
+\ No newline at end of file
diff --git a/debian/patches/001-config.diff b/debian/patches/001-config.diff
new file mode 100644
index 0000000..7ccca33
--- /dev/null
+++ b/debian/patches/001-config.diff
@@ -0,0 +1,115 @@
+Description: add loading config from every path
+ Add loading config from ervery path
+ Separation data / program
+Author: Jörg Frings-Fürst <jff@jff-webhosting.net>
+Last-Update: 2014-05-12
+---
+This patch header follows DEP-3: http://dep.debian.net/deps/dep3/
+Index: trunk/mwc.py
+===================================================================
+--- trunk.orig/mwc.py	2014-09-18 14:47:28.315407312 +0200
++++ trunk/mwc.py	2014-09-18 14:49:11.000000000 +0200
+@@ -305,7 +305,7 @@
+ 
+ if __name__ == "__main__":
+ 
+-        configMod = 'config'
++        configMod = '/etc/mwc/mwc-config'
+         dryrun = None
+ 
+         try:
+@@ -321,9 +321,24 @@
+                         configMod = arg
+                 elif opt in ('-d', '--dry-run'):
+                         dryrun = arg
+-
+-        config = importlib.import_module(configMod)
+-
++	# 
++	# add code to load config from nonsystem path
++	# and change to datadir
++	#
++        try:
++            path = os.path.dirname(configMod)
++            fullname = os.path.basename(configMod)
++            sys.path.append(path)
++            config = importlib.import_module(fullname)
++        except: 
++            print('Error: loading config')
++            sys.exit(2)
++        try:
++            os.chdir(config.datadir)
++        except: 
++            print('Error: datadir not found')
++            sys.exit(3)
++    
+         if dryrun:
+                 for site in config.sites:
+                         if site['shortname'] == dryrun:
+Index: trunk/config_template.py
+===================================================================
+--- trunk.orig/config_template.py	2014-09-18 14:47:28.315407312 +0200
++++ trunk/config_template.py	2014-09-18 14:47:28.271406411 +0200
+@@ -1,5 +1,3 @@
+-import os.path
+-
+ # Copyright: (2013-2014) Michael Till Beck <Debianguru@gmx.de>
+ # License: GPL-2.0+
+ 
+@@ -44,11 +42,11 @@
+ smtphost = 'mysmtpprovider.com'
+ useTLS = True
+ smtpport = 587
+-smtpusername = sender
++smtpusername = 'sender'
+ smtppwd = 'mypassword'
+ receiver = 'me2@mymail.com'
+ 
+-os.chdir('/var/cache/mwc')
++datadir'/var/cache/mwc'
+ 
+ enableRSSFeed = True
+ rssfile = 'feed.xml'
+Index: trunk/mwcfeedserver.py
+===================================================================
+--- trunk.orig/mwcfeedserver.py	2014-09-18 14:47:28.315407312 +0200
++++ trunk/mwcfeedserver.py	2014-09-18 14:49:24.125775123 +0200
+@@ -6,6 +6,7 @@
+ import http.server
+ import socketserver
+ import importlib
++import os
+ import sys
+ import getopt
+ 
+@@ -31,13 +32,26 @@
+         elif opt in ('-p', '--port'):
+                 port = int(arg)
+ 
+-config = importlib.import_module(configMod)
+-
++#
++# add code to load config from nonsystem path
++# and change to datadir
++#
++try:
++	path = os.path.dirname(configMod)
++	fullname = os.path.basename(configMod)
++	sys.path.append(path)
++	config = importlib.import_module(fullname)
++except:
++	print('Error: loading config')
++	sys.exit(2)
+ 
+ handler = http.server.SimpleHTTPRequestHandler
+ 
+ httpd = socketserver.TCPServer((bind, port), handler)
+ 
+ print('Bond to ' + bind + ', listening on port ' + str(port))
+-httpd.serve_forever()
+-
++try:
++	httpd.serve_forever()
++except KeyboardInterrupt:
++	pass
++httpd.server_close()
diff --git a/debian/patches/002-try_mail.diff b/debian/patches/002-try_mail.diff
new file mode 100644
index 0000000..2372d76
--- /dev/null
+++ b/debian/patches/002-try_mail.diff
@@ -0,0 +1,51 @@
+Description: try / except around mail functions
+ add try / except around mail functions to
+ prevent python errors messages
+Author: Jörg Frings-Fürst <debian@jff-webhosting.net>
+Forwarded: via mail
+Applied-Upstream: <version|URL|commit, identifies patches merged upstream, optional>
+Reviewed-by: 
+Last-Update: 2014-05-22
+---
+This patch header follows DEP-3: http://dep.debian.net/deps/dep3/
+Index: trunk/mwc.py
+===================================================================
+--- trunk.orig/mwc.py	2014-05-22 08:43:11.758523862 +0200
++++ trunk/mwc.py	2014-05-22 08:50:09.336051435 +0200
+@@ -212,15 +212,27 @@
+         mail['Subject'] = Header(subject, defaultEncoding)
+ 
+         # initialize session once, not each time this method gets called
+-        if mailsession is None:
+-                mailsession = smtplib.SMTP(config.smtphost, config.smtpport)
+-                if config.useTLS:
+-                        mailsession.ehlo()
+-                        mailsession.starttls()
+-                mailsession.login(config.smtpusername, config.smtppwd)
+-
+-        mailsession.sendmail(config.sender, receiver.split(','), mail.as_string())
+-
++        #
++        # add try / except to open mailsession
++        #
++        try:
++    		if mailsession is None:
++            		mailsession = smtplib.SMTP(config.smtphost, config.smtpport)
++            		if config.useTLS:
++                    		mailsession.ehlo()
++                    		mailsession.starttls()
++            		mailsession.login(config.smtpusername, config.smtppwd)
++        #
++        # add try / except to send mail
++        #
++        except:
++    		printf('Error: Open smtp-session')
++    		exit(4)
++    	try:
++    		mailsession.sendmail(config.sender, receiver.split(','), mail.as_string())
++        except:
++    		printf('Error: sendmail')
++    		exit(5)
+ 
+ # returns a list of all content that is stored locally for a specific site
+ def getFileContents(shortname):
diff --git a/debian/patches/003-syslog.diff b/debian/patches/003-syslog.diff
new file mode 100644
index 0000000..4cab507
--- /dev/null
+++ b/debian/patches/003-syslog.diff
@@ -0,0 +1,96 @@
+Description: add syslog messages on errors
+Author: Jörg Frings-Fürst <debian@jffwebhosting.net>
+Forwarded: via mail
+Last-Update: 2014-05-22
+---
+This patch header follows DEP-3: http://dep.debian.net/deps/dep3/
+Index: trunk/mwc.py
+===================================================================
+--- trunk.orig/mwc.py	2014-09-18 14:47:28.427409601 +0200
++++ trunk/mwc.py	2014-09-18 14:47:28.411409275 +0200
+@@ -19,6 +19,7 @@
+ import sys
+ import getopt
+ import traceback
++import syslog
+ 
+ import subprocess
+ 
+@@ -214,25 +215,28 @@
+         # initialize session once, not each time this method gets called
+         #
+         # add try / except to open mailsession
+-        #
++	#
++	
+         try:
+-    		if mailsession is None:
+-            		mailsession = smtplib.SMTP(config.smtphost, config.smtpport)
+-            		if config.useTLS:
+-                    		mailsession.ehlo()
+-                    		mailsession.starttls()
+-            		mailsession.login(config.smtpusername, config.smtppwd)
+-        #
++                if mailsession is None:
++                        mailsession = smtplib.SMTP(config.smtphost, config.smtpport)
++                        if config.useTLS:
++                                mailsession.ehlo()
++                                mailsession.starttls()
++                        mailsession.login(config.smtpusername, config.smtppwd)
++        except:
++                printf('Error: Open smtp-session')
++                syslog.syslog(syslog.LOG_ERR, 'can not open smtp session')
++                exit(4)
++	#
+         # add try / except to send mail
+         #
++        try:
++                mailsession.sendmail(config.sender, receiver.split(','), mail.as_string())
+         except:
+-    		printf('Error: Open smtp-session')
+-    		exit(4)
+-    	try:
+-    		mailsession.sendmail(config.sender, receiver.split(','), mail.as_string())
+-        except:
+-    		printf('Error: sendmail')
+-    		exit(5)
++                printf('Error: sendmail')
++                syslog.syslog(syslog.LOG_ERR, 'error on sendmail')
++                exit(5)
+ 
+ # returns a list of all content that is stored locally for a specific site
+ def getFileContents(shortname):
+@@ -319,7 +323,11 @@
+ 
+         configMod = '/etc/mwc/mwc-config'
+         dryrun = None
+-
++        
++        #
++        # add syslog open
++        #
++        syslog.openlog()
+         try:
+                 opts, args = getopt.getopt(sys.argv[1:], 'hc:d:', ['help', 'config=', 'dry-run='])
+         except getopt.GetoptError:
+@@ -344,11 +352,13 @@
+             config = importlib.import_module(fullname)
+         except: 
+             print('Error: loading config')
++            syslog.syslog(syslog.LOG_ERR, 'can not found / load mwc-config')
+             sys.exit(2)
+         try:
+             os.chdir(config.datadir)
+         except: 
+             print('Error: datadir not found')
++            syslog.syslog(syslog.LOG_ERR, 'datadir not found')
+             sys.exit(3)
+     
+         if dryrun:
+@@ -370,3 +380,5 @@
+                         mailsession.quit()
+                         mailsession = None
+ 
++        syslog.closelog()
++	
+\ No newline at end of file
diff --git a/debian/patches/series b/debian/patches/series
new file mode 100644
index 0000000..75c3c95
--- /dev/null
+++ b/debian/patches/series
@@ -0,0 +1,3 @@
+001-config.diff
+002-try_mail.diff
+003-syslog.diff
diff --git a/debian/rules b/debian/rules
new file mode 100755
index 0000000..f3956ec
--- /dev/null
+++ b/debian/rules
@@ -0,0 +1,10 @@
+#!/usr/bin/make -f
+
+%:
+	dh $@ --with python3
+
+override_dh_installdeb:
+	mv debian/mwc/usr/share/doc/mwc/examples/config_template.py debian/mwc/usr/share/doc/mwc/examples/mwc-config_template.py
+	mv debian/mwc/usr/share/doc/mwc/examples/mwc.cr debian/mwc/usr/share/doc/mwc/examples/mwc
+	dh_python3
+	dh_installdeb
diff --git a/debian/source/format b/debian/source/format
new file mode 100644
index 0000000..163aaf8
--- /dev/null
+++ b/debian/source/format
@@ -0,0 +1 @@
+3.0 (quilt)
diff --git a/debian/source/include-binaries b/debian/source/include-binaries
new file mode 100644
index 0000000..0957050
--- /dev/null
+++ b/debian/source/include-binaries
@@ -0,0 +1,2 @@
+debian/doc/html/images/bkg.png
+debian/doc/html/images/blacktocat.png
diff --git a/debian/watch b/debian/watch
new file mode 100644
index 0000000..2617459
--- /dev/null
+++ b/debian/watch
@@ -0,0 +1,2 @@
+version=3
+https://github.com/Debianguru/MailWebsiteChanges/releases /Debianguru/MailWebsiteChanges/archive/v(.+)\.tar\.gz
author	Jörg Frings-Fürst <debian@jff-webhosting.net>	2014-09-19 08:49:38 +0200
committer	Jörg Frings-Fürst <debian@jff-webhosting.net>	2014-09-19 08:49:38 +0200
commit	e8136364adf237f9050b7c150c230abd38fff0c0 (patch)
tree	ac80b0b6a998cc1d7c3af2440588a9c113deb68b
parent	b329a335ad10648e64b3de049c083bc04638a072 (diff)