Compare commits

...

52 Commits

Author SHA1 Message Date
874dd95a25 keep DEFAUL_MODE in config.php file 2024-06-04 20:46:17 +02:00
705f4f71c5 update version number 2024-04-08 16:47:57 +02:00
333402a9f2 enhance checkShrinker 2024-04-07 18:31:19 +02:00
e0f1e8f2f7 share config 2024-04-07 17:50:16 +02:00
0f38d26b7b fix bash error in filter.sh 2024-04-07 17:38:48 +02:00
928153cd3d change domain var 2024-04-07 15:27:43 +02:00
b046089d96 lowercase in MIME for variables name*1*, name*2*... 2024-04-06 17:03:08 +02:00
331db8370d fix bug tolowercase in base64 headers 2024-04-06 16:43:01 +02:00
511e14785b Case-insensitive MIME headers 2024-02-11 16:06:02 +01:00
033c497d41 update dockerfile 2024-01-03 08:18:12 +01:00
4c0af57e01 create docker image 2024-01-03 07:55:39 +01:00
a84af46ede create server 2024-01-02 08:22:17 +01:00
f48d5b7a25 default filter = both 2023-10-29 17:07:28 +01:00
6afbaf1fb2 fix BITH 2023-10-29 17:03:58 +01:00
fab
c2656fd377 filtrage sur les domaines
pas de dépollution pour les domaines présents dans /tmp/docker-mailserver/file_domaines_non_depollues.txt
2023-10-23 12:27:40 +02:00
eb5b3b3ec7 fix no space after token in MIME header 2023-09-17 18:36:25 +02:00
dae4379ad7 fix2 filterTest.sh 2023-09-17 17:20:08 +02:00
35a22de854 fix filterTest.sh parameters 2023-09-17 17:11:22 +02:00
3533111a69 update version 2023-04-23 18:13:04 +02:00
3a6b4b3a90 fix "first character of the attachment name was missing" 2023-04-23 17:56:03 +02:00
dce312c33e retry upload 2023-03-03 15:22:13 +01:00
f6db4af4fc upgrade Jirafeau 4.5 2023-01-28 08:29:40 +01:00
f51dc3f7a3 upgrade filterTest (exit code) 2023-01-23 14:09:08 +01:00
2980767d53 update version 2023-01-22 07:27:35 +01:00
bb737e00f7 fix Rainllop 2023-01-22 07:26:24 +01:00
cf7c4102af fix LOG in filter.sh 2023-01-21 19:35:25 +01:00
29ba4fd0d6 fix comma in filename (curl param) 2023-01-16 17:30:37 +01:00
d172596a5d same remove special char in filterTest.sh 2023-01-13 17:08:11 +01:00
33f539a3bb remove special char in password 2023-01-13 17:08:24 +01:00
fc0455e7a8 fix version 2023-01-01 00:08:54 +01:00
e3bbc5ddde fix base64 (==) 2023-01-01 00:07:36 +01:00
38085291ea fix base42 (2) 2022-12-28 15:16:42 +01:00
9141adfb5b fix base64 2022-12-28 15:03:41 +01:00
c9ab7d4f49 fix no content-type in source mbox 2022-12-27 14:48:12 +01:00
18d1214681 update attach filename 2022-12-27 12:01:04 +01:00
285792acfc fix curl filter / fix BOTH 2022-12-27 10:14:47 +01:00
53c5c29a14 fix filter log 2022-12-25 08:49:27 +01:00
61bdc3a4ba fix rfc2047 / filter log / filterTest options 2022-12-25 06:57:44 +01:00
596ae82fe4 fix PGP (skip signed message) 2022-12-23 11:39:44 +01:00
694570a454 update filterTest.sh 2022-12-21 19:54:44 +01:00
122788c2ac fix TAG KAZ_PLAIN_STOP 2022-12-19 18:01:08 +01:00
49a339bdd8 update version 2022-12-17 08:59:10 +01:00
fbaaf6ea3d fix unexpected boundary not in multipart 2022-12-17 08:47:22 +01:00
df7a25d331 double remove & / display version 2022-12-09 08:54:02 +01:00
8e74856ad2 update version 2022-12-08 16:55:12 +01:00
1540688c18 fix double mime, windows \r 2022-12-08 16:42:20 +01:00
51865cfce2 add MODE "none" 2022-12-01 07:47:25 +01:00
b31762ea43 fix quoteted attach filename 2022-11-27 15:46:15 +01:00
d6e167b83b Attachment mode 2022-11-26 21:55:05 +01:00
9c07023316 change filter limit 2022-11-20 15:10:55 +01:00
4027e16004 update version 2022-11-20 15:09:07 +01:00
8efa333b27 merge develpp 2022-11-20 15:02:29 +01:00
26 changed files with 4496 additions and 765 deletions

74
Dockerfile Normal file
View File

@ -0,0 +1,74 @@
# docker build -t depollueur . -f ./docker/Dockerfile
# two stage building
# 1) install compiler and compile filter
# 2) copy filter and install postfix
# Doxkerfile patern from https://vsupalov.com/cache-docker-build-dependencies-without-volume-mounting/
FROM debian as intermediate_depollueur
########################################
RUN apt-get update && apt-get -y autoremove
RUN apt-get -y install locales locales-all
RUN sed -i '/fr_FR.UTF-8/s/^# //g' /etc/locale.gen && locale-gen
ENV LC_ALL fr_FR.UTF-8
ENV LANG fr_FR.UTF-8
ENV LANGUAGE fr_FR:fr
RUN update-locale LANG=fr_FR.UTF-8
RUN apt-get -y install emacs elpa-php-mode apg
RUN apt-get -y install apg dos2unix
# creation du user filter,son repertoire home, copie des fichiers
RUN mkdir /home/filter ; useradd -d /home/filter filter ; chown filter /home/filter
########## >>> ce qui suit va être jetté
RUN apt-get install -y --fix-missing doxygen git \
build-essential make g++ libboost-program-options-dev libboost-system-dev libboost-filesystem-dev libcurl4-gnutls-dev libssl-dev
WORKDIR /home/
RUN git clone https://git.kaz.bzh/KAZ/depollueur.git
WORKDIR /home/depollueur/
RUN make
RUN cp build/out/* /home/filter/
RUN cp src/bash/* /home/filter/
########## <<< on ne garde que le répertoire ci-dessous
##########################################################################
# ###################################################################### #
# # # #
# # On jette tous ce qui est au-dessus pour ne garder que /home/filter # #
# # # #
# ###################################################################### #
##########################################################################
FROM debian
########################################
RUN apt-get update && apt-get -y autoremove
RUN apt-get -y install locales locales-all
RUN sed -i '/fr_FR.UTF-8/s/^# //g' /etc/locale.gen && locale-gen
ENV LC_ALL fr_FR.UTF-8
ENV LANG fr_FR.UTF-8
ENV LANGUAGE fr_FR:fr
RUN update-locale LANG=fr_FR.UTF-8
RUN apt-get -y install emacs elpa-php-mode apg
RUN apt-get -y install apg dos2unix
# creation du user filter,son repertoire home, copie des fichiers
RUN mkdir /home/filter ; useradd -d /home/filter filter ; chown filter /home/filter
########## >>> On fait excatement la même chose que la première fois *
RUN apt-get -y install libboost-program-options-dev libboost-system-dev libboost-filesystem-dev libcurl4-gnutls-dev
########## pour profiter du cahe des couche de docker
COPY --from=intermediate_depollueur /home/filter /home/filter
########## <<< mais cette fois on n'installe pas le compilo
RUN chown filter /home/filter/*; chmod a+rx /home/filter/*
# pour le confort : modif du .bashrc de root
RUN sed -i 's/# alias/alias/g' /root/.bashrc
RUN mkdir /var/log/mail ; chmod a+wrx /var/log/mail
EXPOSE 8080
VOLUME [ "/var/log/mail" ]
ENTRYPOINT [ "/home/filter/server" ]
#HEALTHCHECK --interval=10s --timeout=5s --start-period=20s \
# CMD XXX || exit 1

View File

@ -44,17 +44,23 @@ OUT_DIR = $(BLD_DIR)/out
LIB_DIR = $(BLD_DIR)/lib LIB_DIR = $(BLD_DIR)/lib
OBJ_DIR = $(BLD_DIR)/obj OBJ_DIR = $(BLD_DIR)/obj
KAZ_PRG = eMailShrinker EMS_PRG = eMailShrinker
KAZ_MOD = eMailShrinker EmbeddedData Attachment MainAttachment SizeArg kazDebug kazMisc EMS_MOD = eMailShrinker EmbeddedData Attachment MainAttachment SizeArg kazDebug kazMisc
KAZ_SRC = $(patsubst %, $(CPP_DIR)/%.cpp, $(KAZ_MOD)) EMS_SRC = $(patsubst %, $(CPP_DIR)/%.cpp, $(EMS_MOD))
KAZ_OBJ = $(patsubst %, $(OBJ_DIR)/%.o, $(KAZ_MOD)) EMS_OBJ = $(patsubst %, $(OBJ_DIR)/%.o, $(EMS_MOD))
KAZ_OUT = $(patsubst %, $(OUT_DIR)/%, $(KAZ_PRG)) EMS_OUT = $(patsubst %, $(OUT_DIR)/%, $(EMS_PRG))
JIR_PRG = jirafeauAPI SRV_PRG = server
JIR_MOD = jirafeauAPI SizeArg kazDebug kazMisc SRV_MOD = server kazDebug kazMisc
JIR_SRC = $(patsubst %, $(CPP_DIR)/%.cpp, $(JIR_MOD)) SRV_SRC = $(patsubst %, $(CPP_DIR)/%.cpp, $(SRV_MOD))
JIR_OBJ = $(patsubst %, $(OBJ_DIR)/%.o, $(JIR_MOD)) SRV_OBJ = $(patsubst %, $(OBJ_DIR)/%.o, $(SRV_MOD))
JIR_OUT = $(patsubst %, $(OUT_DIR)/%, $(JIR_PRG)) SRV_OUT = $(patsubst %, $(OUT_DIR)/%, $(SRV_PRG))
TSRV_PRG = testServerRW
TSRV_MOD = testServerRW kazDebug
TSRV_SRC = $(patsubst %, $(CPP_DIR)/%.cpp, $(TSRV_MOD))
TSRV_OBJ = $(patsubst %, $(OBJ_DIR)/%.o, $(TSRV_MOD))
TSRV_OUT = $(patsubst %, $(OUT_DIR)/%, $(TSRV_PRG))
## FLAGS ############################### ## FLAGS ###############################
@ -71,17 +77,28 @@ $(OBJ_DIR)/%.o: $(SRC_DIR)/*/%.cpp
$(CC) $< $(IFLAGS) -cpp -c -o $@ $(CC) $< $(IFLAGS) -cpp -c -o $@
## ENTRIES ############################# ## ENTRIES #############################
all: init eMailShrinker jirafeauAPI all: init $(EMS_PRG) $(SRV_PRG) $(TSRV_PRG) doc
eMailShrinker: $(KAZ_OUT) $(EMS_PRG): $(EMS_OUT)
$(KAZ_OUT): $(KAZ_OBJ) $(EMS_OUT): $(EMS_OBJ)
$(CC) $(KAZ_OBJ) $(IFLAGS) -cpp -L$(LIB_DIR) $(LFLAGS) -o $@ $(CC) $(EMS_OBJ) $(IFLAGS) -cpp -L$(LIB_DIR) $(LFLAGS) -o $@
jirafeauAPI: $(JIR_OUT) $(SRV_PRG): $(SRV_OUT)
$(JIR_OUT): $(JIR_OBJ) $(SRV_OUT): $(SRV_OBJ)
$(CC) $(JIR_OBJ) $(IFLAGS) -cpp -L$(LIB_DIR) $(LFLAGS) -o $@ $(CC) $(SRV_OBJ) $(IFLAGS) -cpp -L$(LIB_DIR) $(LFLAGS) -o $@
$(TSRV_PRG): $(TSRV_OUT)
$(TSRV_OUT): $(TSRV_OBJ)
$(CC) $(TSRV_OBJ) $(IFLAGS) -cpp -L$(LIB_DIR) $(LFLAGS) -o $@
image:
docker build -t depollueur . -f ./Dockerfile
doc:
doxygen src/Doxyfile
init: init:
mkdir -p $(OUT_DIR) $(OBJ_DIR) $(LIB_DIR) mkdir -p $(OUT_DIR) $(OBJ_DIR) $(LIB_DIR)
@ -94,8 +111,8 @@ wipe: clean
-rm -rf $(OUT_DIR) $(LIB_DIR) $(BLD_DIR) -rm -rf $(OUT_DIR) $(LIB_DIR) $(BLD_DIR)
## DEPENDS ############################# ## DEPENDS #############################
ALL_OUT = $(KAZ_PRG) $(JIR_PRG) ALL_OUT = $(EMS_PRG) $(SRV_PRG) $(TSRV_PRG)
ALL_OBJ = $(KAZ_OBJ) $(JIR_OBJ) ALL_OBJ = $(EMS_OBJ) $(SRV_OBJ) $(TSRV_OBJ)
DEPENDS = ${ALL_OUT:=.d} ${ALL_OBJ:.o=.d} DEPENDS = ${ALL_OUT:=.d} ${ALL_OBJ:.o=.d}
-include ${DEPENDS} -include ${DEPENDS}

View File

@ -13,7 +13,6 @@ depollueur/
│   ├── Attachment.cpp │   ├── Attachment.cpp
│   ├── eMailShrinker.cpp │   ├── eMailShrinker.cpp
│   ├── EmbeddedData.cpp │   ├── EmbeddedData.cpp
│   ├── jirafeauAPI.cpp
│   ├── kazDebug.cpp │   ├── kazDebug.cpp
│   ├── kazMisc.cpp │   ├── kazMisc.cpp
│   ├── MainAttachment.cpp │   ├── MainAttachment.cpp
@ -33,7 +32,7 @@ depollueur/
## Compilation ## Compilation
```bash ```bash
sudo apt-get install --fix-missing build-essential make g++ libboost-program-options-dev libboost-system-dev libboost-filesystem-dev libcurl4-gnutls-dev libssl-dev sudo apt-get install --fix-missing build-essential make g++ libboost-program-options-dev libboost-system-dev libboost-filesystem-dev libcurl4-gnutls-dev libssl-dev doxygen dos2unix
git clone https://git.kaz.bzh/KAZ/depollueur.git git clone https://git.kaz.bzh/KAZ/depollueur.git
# or for contributors : # or for contributors :

18
docker-compose.yml Normal file
View File

@ -0,0 +1,18 @@
version: '3.3'
services:
depollueur:
# ports:
# - 8088:80
image: depollueur
container_name: depollueurServ
# restart: ${restartPolicy}
volumes:
- mailLog:/var/log/mail
- /etc/localtime:/etc/localtime:ro
- /etc/timezone:/etc/timezone:ro
volumes:
mailLog:
# config:

2658
src/Doxyfile Normal file

File diff suppressed because it is too large Load Diff

View File

@ -2,6 +2,7 @@
/* /*
* Kaz addon (see https://git.kaz.bzh/KAZ/depollueur for information) * Kaz addon (see https://git.kaz.bzh/KAZ/depollueur for information)
* create un archive for a set of file or update file deadline * create un archive for a set of file or update file deadline
* version : 2.21 (2024-06-04)
a.php?r=email => track a.php?r=email => track
a.php?p=email => period a.php?p=email => period
@ -25,11 +26,14 @@ require (JIRAFEAU_ROOT . 'lib/functions.php');
require (JIRAFEAU_ROOT . 'lib/lang.php'); require (JIRAFEAU_ROOT . 'lib/lang.php');
define ('VAR_TOKENS', $cfg ['var_root'].'tokens/'); define ('VAR_TOKENS', $cfg ['var_root'].'tokens/');
define ('VAR_MODE', $cfg ['var_root'].'mode/');
define ('VAR_TRACKS', $cfg ['var_root'].'tracks/'); define ('VAR_TRACKS', $cfg ['var_root'].'tracks/');
define ('VAR_LANG', $cfg ['var_root'].'lang/');
define ('VAR_PERIOD', $cfg ['var_root'].'period/'); define ('VAR_PERIOD', $cfg ['var_root'].'period/');
define ('VAR_LANG', $cfg ['var_root'].'lang/');
define ('VAR_FAKE', $cfg ['var_root'].'fake/'); define ('VAR_FAKE', $cfg ['var_root'].'fake/');
define ('VAR_ADMIN', $cfg ['var_root'].'admin/'); define ('VAR_ADMIN', $cfg ['var_root'].'admin/');
define ('VAR_CONFIG', $cfg ['var_root'].'config/');
define ('FILE_CONFIG', VAR_CONFIG.'default.php');
$domain="kaz.local"; $domain="kaz.local";
if (preg_match ("%^.*//([^/]*)/?.*$%", $cfg ['web_root'], $matches)) if (preg_match ("%^.*//([^/]*)/?.*$%", $cfg ['web_root'], $matches))
@ -39,8 +43,19 @@ define ('MAX_VALID_UPLOAD_TIME', 60);
define ('TOKEN_USE_LIMIT', "-2 hours"); define ('TOKEN_USE_LIMIT', "-2 hours");
define ('TOKEN_LOGIN_LIMIT', "-15 minutes"); define ('TOKEN_LOGIN_LIMIT', "-15 minutes");
define ('TOKEN_LOGOUT_LIMIT', "-8 hours"); define ('TOKEN_LOGOUT_LIMIT', "-8 hours");
define ('DEFAULT_PERIOD', "month"); if (!file_exists (VAR_CONFIG))
define ('DEFAULT_LANG', "fr"); mkdir (VAR_CONFIG, 0755);
if (!file_exists (FILE_CONFIG)) {
file_put_contents (FILE_CONFIG, "<?php".NL.
"/* if error with DEFAULT_MODE, DEFAULT_PERIOD or DEFAULT_LANG then remove this file. */".NL.
"define ('DEFAULT_MODE', 'footer');".NL.
"define ('DEFAULT_PERIOD', 'month');".NL.
"define ('DEFAULT_LANG', 'fr');".NL.NL);
define ('DEFAULT_MODE', 'footer');
define ('DEFAULT_PERIOD', 'month');
define ('DEFAULT_LANG', 'fr');
} else
require (FILE_CONFIG);
define ('E_BAD_ARCHIVE_NAME', 'Bad archive name format'); define ('E_BAD_ARCHIVE_NAME', 'Bad archive name format');
define ('E_CREATE_ZIP', "Impossible de cr&eacute;er l'archive."); define ('E_CREATE_ZIP', "Impossible de cr&eacute;er l'archive.");
@ -75,13 +90,14 @@ define ('M_WELCOME', "<p>Informations concernant le compte : <b>___SENDER___</b>
define ('M_INCONSISTENT_DATES', define ('M_INCONSISTENT_DATES',
" (dates incoh&eacute;antes avec ___FILENAME___ : ___DIRTIME___ != ___FILETIME___)"); " (dates incoh&eacute;antes avec ___FILENAME___ : ___DIRTIME___ != ___FILETIME___)");
define ('A_ACTION', 'a'); // action : T_LOGIN, T_LOGOUT, A_RECORD+(on|off), A_LANG(fr|en|br), A_PERIOD(minute|hour|day|week|month|quarter) define ('A_ACTION', 'a'); // action : T_LOGIN, T_LOGOUT, A_MODE(none|footer|attachment|both), A_RECORD+(on|off), A_PERIOD(minute|hour|day|week|month|quarter), A_LANG(fr|en|br)
define ('A_GET', 'g'); // get archive define ('A_GET', 'g'); // get archive
define ('A_HASH', 'h'); // file to update or delete define ('A_HASH', 'h'); // file to update or delete
define ('A_OPEN_TOKEN', 'o'); // ask token define ('A_OPEN_TOKEN', 'o'); // ask token
define ('A_SENDER', 's'); // session sender define ('A_SENDER', 's'); // session sender
define ('A_TOKEN', 't'); // session token define ('A_TOKEN', 't'); // session token
define ('A_UPDATE', 'u'); // update perriod for file or archive define ('A_UPDATE', 'u'); // update perriod for file or archive
define ('A_MODE', 'm'); // get mode status
define ('A_RECORD', 'r'); // get track status define ('A_RECORD', 'r'); // get track status
define ('A_PERIOD', 'p'); // get period status define ('A_PERIOD', 'p'); // get period status
define ('A_LANG', 'l'); // get lang status define ('A_LANG', 'l'); // get lang status
@ -111,7 +127,8 @@ define ('T_ARCHIVE_TITLE', "archive_content");
define ('T_ARCHIVE_MIME', "text/kaz_email_archive"); define ('T_ARCHIVE_MIME', "text/kaz_email_archive");
$langText = ['fr' => "Francais", 'br' => "Breton", 'en' => "english"]; $modeText = ['none' => "sans", 'footer' => "pied de page", 'attachment' => "pi&egrave;ce jointe", 'both' => "les deux"];
$trackText = ['on' => "oui", 'off' => "non"];
$periodText = ['minute' => "minute", 'hour' => "heure", 'day' => "jour", 'week' => "semaine", 'month' => "mois"]; $periodText = ['minute' => "minute", 'hour' => "heure", 'day' => "jour", 'week' => "semaine", 'month' => "mois"];
// XXX , 'quarter' => "trimestre"]; // XXX , 'quarter' => "trimestre"];
$periodButton = ['hour' => ["&#128341;", ">1 heure"], $periodButton = ['hour' => ["&#128341;", ">1 heure"],
@ -119,7 +136,7 @@ $periodButton = ['hour' => ["&#128341;", ">1 heure"],
'week' => ["&#128349;", "> 1 semaine"], 'week' => ["&#128349;", "> 1 semaine"],
'month' => ["&#128350;", "> 1 mois"]]; 'month' => ["&#128350;", "> 1 mois"]];
// XXX 'quarter' => ["&#128351;", "> 1 trimestre"]]; // XXX 'quarter' => ["&#128351;", "> 1 trimestre"]];
$trackText = ['on' => "oui", 'off' => "non"]; $langText = ['fr' => "Francais", 'br' => "Breton", 'en' => "english"];
$doLogout = ''; $doLogout = '';
$message = ''; $message = '';
@ -130,6 +147,19 @@ $message = '';
/* Remove errors. */ /* Remove errors. */
@error_reporting (0); @error_reporting (0);
// ========================================
if (isset ($_REQUEST [A_MODE]) && !empty ($_REQUEST [A_MODE])) {
if (!preg_match ("/^([a-z0-9\+_\-]+)(\.[a-z0-9\+_\-]+)*@([a-z0-9\-]+\.)+[a-z]{2,6}$/i", $_REQUEST [A_MODE]))
$content = DEFAULT_MODE.NL;
else
$content = getSenderMode ($_REQUEST [A_MODE]).NL;
header ('HTTP/1.0 200 OK');
header ('Content-Length: ' . strlen ($content));
header ('Content-Type: text/plain');
echo $content;
exit;
}
// ======================================== // ========================================
if (isset ($_REQUEST [A_RECORD]) && !empty ($_REQUEST [A_RECORD])) { if (isset ($_REQUEST [A_RECORD]) && !empty ($_REQUEST [A_RECORD])) {
if (!preg_match ("/^([a-z0-9\+_\-]+)(\.[a-z0-9\+_\-]+)*@([a-z0-9\-]+\.)+[a-z]{2,6}$/i", $_REQUEST [A_RECORD])) if (!preg_match ("/^([a-z0-9\+_\-]+)(\.[a-z0-9\+_\-]+)*@([a-z0-9\-]+\.)+[a-z]{2,6}$/i", $_REQUEST [A_RECORD]))
@ -193,6 +223,29 @@ function returnError ($msg) {
exit; exit;
} }
// ========================================
function setSenderMode ($sender, $mode) {
if (!$sender)
return;
if (!file_exists (VAR_MODE))
mkdir (VAR_MODE, 0755);
if (empty ($mode) || DEFAULT_MODE == $mode) {
rmSenderMode ($sender);
} else
file_put_contents (VAR_MODE.$sender, $mode.NL);
}
function rmSenderMode ($sender) {
if (!$sender)
return;
if (file_exists (VAR_MODE.$sender))
unlink (VAR_MODE.$sender);
}
function getSenderMode ($sender) {
if ($sender && file_exists (VAR_MODE.$sender))
return trim (file (VAR_MODE.$sender)[0]);
return DEFAULT_MODE;
}
// ======================================== // ========================================
function setSenderTrack ($sender) { function setSenderTrack ($sender) {
if (!$sender) if (!$sender)
@ -211,29 +264,6 @@ function isSenderTrack ($sender) {
return $sender && file_exists (VAR_TRACKS.$sender); return $sender && file_exists (VAR_TRACKS.$sender);
} }
// ========================================
function setSenderLang ($sender, $lang) {
if (!$sender)
return;
if (!file_exists (VAR_LANG))
mkdir (VAR_LANG, 0755);
if (empty ($lang) || DEFAULT_LANG == $lang) {
rmSenderLang ($sender);
} else
file_put_contents (VAR_LANG.$sender, $lang.NL);
}
function rmSenderLang ($sender) {
if (!$sender)
return;
if (file_exists (VAR_LANG.$sender))
unlink (VAR_LANG.$sender);
}
function getSenderLang ($sender) {
if ($sender && file_exists (VAR_LANG.$sender))
return trim (file (VAR_LANG.$sender)[0]);
return DEFAULT_LANG;
}
// ======================================== // ========================================
function setSenderPeriod ($sender, $period) { function setSenderPeriod ($sender, $period) {
if (!$sender) if (!$sender)
@ -286,6 +316,29 @@ function period2seconds ($periodName) {
} }
} }
// ========================================
function setSenderLang ($sender, $lang) {
if (!$sender)
return;
if (!file_exists (VAR_LANG))
mkdir (VAR_LANG, 0755);
if (empty ($lang) || DEFAULT_LANG == $lang) {
rmSenderLang ($sender);
} else
file_put_contents (VAR_LANG.$sender, $lang.NL);
}
function rmSenderLang ($sender) {
if (!$sender)
return;
if (file_exists (VAR_LANG.$sender))
unlink (VAR_LANG.$sender);
}
function getSenderLang ($sender) {
if ($sender && file_exists (VAR_LANG.$sender))
return trim (file (VAR_LANG.$sender)[0]);
return DEFAULT_LANG;
}
// ======================================== // ========================================
function setSenderFake ($error, $sender, $owner, $dirLink, $fileLink) { function setSenderFake ($error, $sender, $owner, $dirLink, $fileLink) {
global $doLogout; global $doLogout;
@ -976,6 +1029,10 @@ if ($doLogout || (isset ($_REQUEST [A_ACTION]) && $_REQUEST [A_ACTION] == T_LOGO
if (isset ($_REQUEST [A_ACTION])) { if (isset ($_REQUEST [A_ACTION])) {
// change track // change track
switch (true) { switch (true) {
case preg_match ("/^".A_MODE."(".implode ("|", array_keys ($modeText)).")$/i", $_REQUEST [A_ACTION], $matches):
setSenderMode ($sender, $matches [1]);
$message .= "Votre mode &agrave; &eacute;t&eacute; mise &agrave; jour.";
break;
case preg_match ("/^".A_RECORD."(on|off)$/i", $_REQUEST [A_ACTION], $matches): case preg_match ("/^".A_RECORD."(on|off)$/i", $_REQUEST [A_ACTION], $matches):
if ($matches [1] == "on") if ($matches [1] == "on")
setSenderTrack ($sender); setSenderTrack ($sender);
@ -983,14 +1040,14 @@ if (isset ($_REQUEST [A_ACTION])) {
rmSenderTrack ($sender); rmSenderTrack ($sender);
$message .= "Votre suivi &agrave; &eacute;t&eacute; mise &agrave; jour."; $message .= "Votre suivi &agrave; &eacute;t&eacute; mise &agrave; jour.";
break; break;
case preg_match ("/^".A_LANG."(".implode ("|", array_keys ($langText)).")$/i", $_REQUEST [A_ACTION], $matches):
setSenderLang ($sender, $matches [1]);
$message .= "Votre lang &agrave; &eacute;t&eacute; mise &agrave; jour.";
break;
case preg_match ("/^".A_PERIOD."(".implode ("|", array_keys ($periodText)).")$/i", $_REQUEST [A_ACTION], $matches): case preg_match ("/^".A_PERIOD."(".implode ("|", array_keys ($periodText)).")$/i", $_REQUEST [A_ACTION], $matches):
setSenderPeriod ($sender, $matches [1]); setSenderPeriod ($sender, $matches [1]);
$message .= "Votre p&eacute;riode &agrave; &eacute;t&eacute; mise &agrave; jour."; $message .= "Votre p&eacute;riode &agrave; &eacute;t&eacute; mise &agrave; jour.";
break; break;
case preg_match ("/^".A_LANG."(".implode ("|", array_keys ($langText)).")$/i", $_REQUEST [A_ACTION], $matches):
setSenderLang ($sender, $matches [1]);
$message .= "Votre lang &agrave; &eacute;t&eacute; mise &agrave; jour.";
break;
} }
} }
@ -1085,10 +1142,20 @@ div.frame {border: 1px; border-style: solid; padding: 1em; margin: 1em;}
--></style> --></style>
<?php <?php
$defaultChecked = []; $defaultChecked = [];
$defaultChecked [getSenderMode ($sender)] = ' selected="selected"';
$defaultChecked [isSenderTrack ($sender) ? "on" : "off"] = ' checked="checked"'; $defaultChecked [isSenderTrack ($sender) ? "on" : "off"] = ' checked="checked"';
$defaultChecked [getSenderPeriod ($sender)] = ' selected="selected"'; $defaultChecked [getSenderPeriod ($sender)] = ' selected="selected"';
$defaultChecked [getSenderLang ($sender)] = ' selected="selected"'; $defaultChecked [getSenderLang ($sender)] = ' selected="selected"';
echo echo
'<form method="post">'.
'Je veux que mes futurs envois soient d&eacute;pollu&eacute; en pla&ccedil;ant les liens de t&eacute;l&eacute;chargements '.
'<select name="'.A_ACTION.'" style="width: auto !important;">';
foreach ($modeText as $item => $text)
echo ' <option value="'.A_MODE.$item.'"'.$defaultChecked [$item].'>'.$text.'</option>';
echo
'</select> '.
'<button type="submit">'."valider".'</button>'.
'</form>'.
'<form method="post">'. '<form method="post">'.
'Je veux que Kaz suive tous mes futurs envois: '. 'Je veux que Kaz suive tous mes futurs envois: '.
'<input type="hidden" name="'.A_SENDER.'" value="'.$sender.'"/>'. '<input type="hidden" name="'.A_SENDER.'" value="'.$sender.'"/>'.

View File

@ -89,6 +89,8 @@ if (!empty($delete_code) && $delete_code == $link['link_code']) {
'</p></div>'; '</p></div>';
} else { ?> } else { ?>
<div> <div>
<form action="<?php echo 'f.php?h=' . $link_name . '&amp;d=' . $delete_code; ?>" method="post" id="submit_delete_post" class="form login">
<input type="hidden" name="do_delete" value="1" />
<form action="f.php" method="post" id="submit_delete_post" class="form login"> <form action="f.php" method="post" id="submit_delete_post" class="form login">
<input type="hidden" name="do_delete" value=1/> <input type="hidden" name="do_delete" value=1/>
<fieldset> <fieldset>
@ -102,8 +104,6 @@ if (!empty($delete_code) && $delete_code == $link['link_code']) {
</td></tr> </td></tr>
<tr><td> <tr><td>
<input type="submit" id="submit_delete" value="<?php echo t('DELETE'); ?>" <input type="submit" id="submit_delete" value="<?php echo t('DELETE'); ?>"
<?php $action_delete="'f.php?h=' . $link_name . '&amp;d=' . $delete_code'; document.getElementById('submit_delete').submit ();"; ?>
onclick="document.getElementById('submit_delete_post').action='<?php echo $action_delete; ?>" />
</td></tr> </td></tr>
</table> </table>
</fieldset></form></div><?php </fieldset></form></div><?php
@ -145,7 +145,7 @@ if (!empty($link['key'])) {
'<legend>' . t('PSW_PROTEC') . '<legend>' . t('PSW_PROTEC') .
'</legend><table><tr><td>' . '</legend><table><tr><td>' .
t('GIMME_PSW') . ' : ' . t('GIMME_PSW') . ' : ' .
'<input type = "password" name = "key" />' . '<input type = "password" name = "key" autocomplete = "current-password"/>' .
'</td></tr>' . '</td></tr>' .
'<tr><td>' . '<tr><td>' .
t('USING_SERVICE'). ' <a href="tos.php" target="_blank" rel="noopener noreferrer">' . t('TOS') . '</a>.' . t('USING_SERVICE'). ' <a href="tos.php" target="_blank" rel="noopener noreferrer">' . t('TOS') . '</a>.' .
@ -276,12 +276,17 @@ elseif ($link['crypted']) {
} }
/* Read file. */ /* Read file. */
else { else {
if ($cfg['use_xsendfile']) {
$file_web_path = preg_replace('#^' . $_SERVER['DOCUMENT_ROOT'] . '#', '', VAR_FILES);
header('X-Sendfile: ' . $file_web_path . $p . $link['hash']);
} else {
$r = fopen(VAR_FILES . $p . $link['hash'], 'r'); $r = fopen(VAR_FILES . $p . $link['hash'], 'r');
while (!feof($r)) { while (!feof($r)) {
print fread($r, 1024); print fread($r, 1024);
} }
fclose($r); fclose($r);
} }
}
if ($link['onetime'] == 'O') { if ($link['onetime'] == 'O') {
jirafeau_delete_link($link_name); jirafeau_delete_link($link_name);

View File

@ -1,121 +0,0 @@
<?php
define ('JIRAFEAU_ROOT', dirname (__FILE__) . '/');
require (JIRAFEAU_ROOT . 'lib/settings.php');
require (JIRAFEAU_ROOT . 'lib/functions.php');
require (JIRAFEAU_ROOT . 'lib/lang.php');
@set_time_limit (0);
/* Remove errors. */
@error_reporting (0);
if (isset ($_REQUEST ['l']) && !empty ($_REQUEST ['l']))
$linksPass = explode ("/", $_REQUEST ["l"]);
else if (isset ($_REQUEST ['h']) && !empty ($_REQUEST ['h']))
$linksPass = $_REQUEST ["h"];
else
die ("no links");
if (!is_array ($linksPass))
die ("no list is given: ".$h);
$notFoundCount=0;
$map = [];
// First pass: check
foreach ($linksPass as $line) {
if (strpos ($line, '~') !== false)
$couple = explode ("~", $line, 2);
else
$couple = explode ("/", $line, 2);
if (count ($couple) == 0)
continue;
$link_name = $couple [0];
if (!$link_name)
continue;
$crypt_key = count ($couple) == 2 ? $couple [1] : "";
if (!preg_match ('/[0-9a-zA-Z_-]+$/', $link_name))
die ("bad link format : ".$link_name);
$link = jirafeau_get_link ($link_name);
if (count ($link) == 0) {
++$notFoundCount;
continue;
}
$key = $link['key'];
if ($key) {
preg_match ( '/[0-9a-zA-Z_-]+/', $link['key'], $matches);
$key = $matches[1];
}
if ($key && (empty ($crypt_key) || $key != $crypt_key))
die ("bad key for ".$link);
$map [$link_name] = $crypt_key;
}
// second pass: send
if (isset ($_REQUEST ['n']) && !empty ($_REQUEST ['n']))
$dirname=$_REQUEST ['n'];
else
$dirname="kaz-".date ("Ymd-His");
$tmpFileName = tempnam (sys_get_temp_dir (), $dirname."-");
$zip = new ZipArchive;
if (!$zip)
die ("can't create tmp");
if ($zip->open ($tmpFileName.".zip", ZipArchive::CREATE) !== TRUE)
die ("can't create tmp");
if ($notFoundCount) {
$zip->addFromString ($dirname."-Avertissement.txt", $notFoundCount. ($notFoundCount ? " fichier est expiré." : " fichiers sont expirés."));
}
$single_name=[];
foreach ($map as $link_name => $crypt_key) {
$link = jirafeau_get_link ($link_name);
$p = s2p ($link ['hash']);
$src_name = $dst_name = $link['file_name'];
if (in_array ($src_name, $single_name))
for ($i = 0; $i < 10000; ++$i) {
$dst_name = sprintf ("%s-%2d", $src_name, $i);
if (!in_array ($dst_name, $single_name))
break;
}
$single_name[]=$dst_name;
// send
if ($link['crypted']) {
$m = mcrypt_module_open ('rijndael-256', '', 'ofb', '');
$md5_key = md5 ($crypt_key);
$iv = jirafeau_crypt_create_iv ($md5_key, mcrypt_enc_get_iv_size ($m));
mcrypt_generic_init ($m, $md5_key, $iv);
$r = fopen (VAR_FILES . $p . $link['hash'], 'r');
$content = "";
while (!feof ($r)) {
$dec = mdecrypt_generic ($m, fread ($r, 1024));
$content .= $dec;
ob_flush ();
}
fclose ($r);
$zip->addFromString ($dirname."/".$dst_name, $content);
mcrypt_generic_deinit ($m);
mcrypt_module_close ($m);
continue;
}
$zip->addFile (VAR_FILES . $p . $link['hash'], $dirname."/".$dst_name);
}
$zip->close ();
if (!is_file ($tmpFileName.".zip"))
die ("can't retreive tmp");
header ("HTTP/1.0 200 OK");
header ("Content-Type: application/zip");
header ('Content-Disposition: filename="'.$dirname.'.zip"');
$r = fopen($tmpFileName.".zip", 'r');
while (!feof ($r)) {
print fread ($r, 1024);
ob_flush ();
}
fclose ($r);
unlink ($tmpFileName.".zip");
unlink ($tmpFileName);

78
src/bash/checkAltered.sh Executable file
View File

@ -0,0 +1,78 @@
#!/bin/bash
##########################################################################
# Copyright KAZ 2021 #
# #
# contact (at) kaz.bzh #
# #
# This software is a filter to shrink email by attachment extraction. #
# #
# This software is governed by the CeCILL-B license under French law and #
# abiding by the rules of distribution of free software. You can use, #
# modify and/or redistribute the software under the terms of the #
# CeCILL-B license as circulated by CEA, CNRS and INRIA at the following #
# URL "http://www.cecill.info". #
# #
# As a counterpart to the access to the source code and rights to copy, #
# modify and redistribute granted by the license, users are provided #
# only with a limited warranty and the software's author, the holder of #
# the economic rights, and the successive licensors have only limited #
# liability. #
# #
# In this respect, the user's attention is drawn to the risks associated #
# with loading, using, modifying and/or developing or reproducing the #
# software by the user in light of its specific status of free software, #
# that may mean that it is complicated to manipulate, and that also #
# therefore means that it is reserved for developers and experienced #
# professionals having in-depth computer knowledge. Users are therefore #
# encouraged to load and test the software's suitability as regards #
# their requirements in conditions enabling the security of their #
# systems and/or data to be ensured and, more generally, to use and #
# operate it in the same conditions as regards security. #
# #
# The fact that you are presently reading this means that you have had #
# knowledge of the CeCILL-B license and that you accept its terms. #
##########################################################################
PRG=$(basename $0)
usage () {
echo "Usage: ${PRG} logDir"
exit 1
}
BOLD='\e[1m'
RED='\e[0;31m'
GREEN='\e[0;32m'
YELLOW='\e[0;33m'
BLUE='\e[0;34m'
MAGENTA='\e[0;35m'
CYAN='\e[0;36m'
NC='\e[0m' # No Color
NL='
'
[ "$#" -eq 1 ] || usage
logDir=$(realpath "$1")
########################################
# recherche des binaires
cd $(dirname $0)
eMailShrinker="$(realpath "./eMailShrinker")"
[ -x "${eMailShrinker}" ] || eMailShrinker="$(realpath "../../build/out/eMailShrinker")"
[ -x "${eMailShrinker}" ] || ( echo "${RED}eMailShrinker not found${NC}" ; exit)
cd "${logDir}"
for i in in.*.orig;
do
clear
echo -e " ${GREEN}${BOLD}${i//.orig}${NC}\n"
"${eMailShrinker}" -l ${i}
echo -e "\n ####################\n"
"${eMailShrinker}" -l ${i//.orig}.altered
echo -en "\n(q = quit / y = rm / default = continue)? "
read rep
case "${rep}" in
"q" ) break;;
"y") rm ${i} ${i//.orig}.altered
esac
done

99
src/bash/checkShrinker.sh Executable file
View File

@ -0,0 +1,99 @@
#!/bin/bash
##########################################################################
# Copyright KAZ 2021 #
# #
# contact (at) kaz.bzh #
# #
# This software is a filter to shrink email by attachment extraction. #
# #
# This software is governed by the CeCILL-B license under French law and #
# abiding by the rules of distribution of free software. You can use, #
# modify and/or redistribute the software under the terms of the #
# CeCILL-B license as circulated by CEA, CNRS and INRIA at the following #
# URL "http://www.cecill.info". #
# #
# As a counterpart to the access to the source code and rights to copy, #
# modify and redistribute granted by the license, users are provided #
# only with a limited warranty and the software's author, the holder of #
# the economic rights, and the successive licensors have only limited #
# liability. #
# #
# In this respect, the user's attention is drawn to the risks associated #
# with loading, using, modifying and/or developing or reproducing the #
# software by the user in light of its specific status of free software, #
# that may mean that it is complicated to manipulate, and that also #
# therefore means that it is reserved for developers and experienced #
# professionals having in-depth computer knowledge. Users are therefore #
# encouraged to load and test the software's suitability as regards #
# their requirements in conditions enabling the security of their #
# systems and/or data to be ensured and, more generally, to use and #
# operate it in the same conditions as regards security. #
# #
# The fact that you are presently reading this means that you have had #
# knowledge of the CeCILL-B license and that you accept its terms. #
##########################################################################
PRG=$(basename $0)
FILTER_TEST="$(realpath "$(dirname $0)/filterTest.sh")"
ATTACH_MODE="FOOTER"
usage () {
echo "Usage: ${PRG} {NONE|FOOTER|ATTACHMENT|BOTH} mbox..."
exit 1
}
########################################
BOLD='\e[1m'
RED='\e[0;31m'
GREEN='\e[0;32m'
YELLOW='\e[0;33m'
BLUE='\e[0;34m'
MAGENTA='\e[0;35m'
CYAN='\e[0;36m'
GREY='\e[0;90m'
BG_BLACK='\e[0;40m'
BG_RED='\e[0;41m'
BG_GREEN='\e[0;42m'
BG_YELLOW='\e[0;43m'
BG_BLUE='\e[0;44m'
BG_MAGENTA='\e[0;45m'
CYAN='\e[0;36m'
NC='\e[0m' # No Color
NL='
'
########################################
[ "$#" -lt 2 ] && usage
ATTACH_MODE="$1"; shift
case "${ATTACH_MODE}" in
""|NONE|FOOTER|ATTACHMENT|BOTH);;
*) usage;;
esac
start=0
loop=true
while [ ! -z "${loop}" ]; do
loop=""
count=0
for mbox in $*; do
clear
((count=count+1))
(( count < start )) && continue
echo -e " ${GREY}${count}/$# ${GREEN}${BOLD}${mbox}${NC}\n"
"${FILTER_TEST}" -s -m "${ATTACH_MODE}" "${mbox}"
echo -en "\n(q = quit / [0-9]* = goto / default = continue)? "
read rep
case "${rep}" in
"q" ) break;;
[0-9]* )
start="${rep}"
(( count < start )) && continue
loop=true
break;;
esac
done
done

203
src/bash/filter.sh Normal file → Executable file
View File

@ -1,4 +1,4 @@
#!/bin/sh #!/bin/bash
########################################################################## ##########################################################################
# Copyright KAZ 2021 # # Copyright KAZ 2021 #
# # # #
@ -35,90 +35,163 @@
########################################################################## ##########################################################################
# - installer l'utilitaire apg pour génération de mot de passes # - installer l'utilitaire apg pour génération de mot de passes
# - installer l'utilitaire dos2unix
# - le contenu de INSPECT_DIR doit être accessible en écriture pour le # - le contenu de INSPECT_DIR doit être accessible en écriture pour le
# proriétaire du script # proriétaire du script
# - shrinkEMail et jirafeau.sh doivent être accessible en execution pour # - shrinkEMail doit être accessible en execution pour le proriétaire
# le roriétaire du script # - comme le programme n'a pas de privigèle, il faut que root fasse avant :
# mkdir -p "${DIR_LOG}/pb/" ; chmod a+rwx "${DIR_LOG}/pb/"
########################################################################## ##########################################################################
DEFAULT_MODE="both"
DEFAULT_PERIOD="month"
DEFAULT_TRACK=""
cd $(dirname $0) cd $(dirname $0)
DOMAINNAME=$(cat domainname) DOMAINEDEPOT=$(cat config/domainedepot)
# Exit codes from <sysexits.h> # Exit codes from <sysexits.h>
EX_TEMPFAIL=75 EX_TEMPFAIL=75
EX_UNAVAILABLE=69 EX_UNAVAILABLE=69
EX_TOO_LARGE=552 EX_TOO_LARGE=552
INSPECT_DIR=/var/spool/filter INSPECT_DIR=/var/spool/filter
DIR_LOG=/var/log/mail DIR_LOG=/var/log/mail
FIC_LOG=${DIR_LOG}/filter.log FIC_LOG="${DIR_LOG}/filter.log"
TMP_LOG="$(mktemp)"
SENDMAIL="/usr/sbin/sendmail -G -i" SENDMAIL="/usr/sbin/sendmail -G -i"
MAILS=/tmp/FILTER MAILS=/tmp/FILTER
MAX_KEEP_IN_MAIL=5ki MAX_KEEP_IN_MAIL=5ki
MAX_UPLOAD_SIZE=1Gi MAX_UPLOAD_SIZE=1Gi
SHRINK_CMD=/home/filter/eMailShrinker SHRINK_CMD=/home/filter/eMailShrinker
JIRAFEAU_CMD=/home/filter/jirafeauAPI JIRAFEAU_URL=https://depot.${DOMAINEDEPOT:-"kaz.bzh"}
JIRAFEAU_URL=https://depot.${DOMAINNAME:-"kaz.bzh"}
JIRAFEAU_LOCAL=http://depot JIRAFEAU_LOCAL=http://depot
JIRAFEAU_TIME=month
MD5_CMD=/usr/bin/md5sum MD5_CMD=/usr/bin/md5sum
DISCLAMER_CMD=altermime DISCLAMER_CMD=altermime
MAX_FINAL_SIZE=307200 # 300ki MAX_FINAL_SIZE=2097152 # 2Mi
ARCHIVE_TITLE="archive_content" ARCHIVE_TITLE="archive_content"
ARCHIVE_MIME="text/kaz_email_archive" ARCHIVE_MIME="text/kaz_email_archive"
FILE_SKIP_DOMAINS="config/file_domaines_non_depollues.txt"
#on enlève les commentaires et les lignes vides
SKIP_DOMAINS=$(grep -Ev '^#|^[[:space:]]*$' $FILE_SKIP_DOMAINS 2>/dev/null)
KEEP_FAILED=true KEEP_FAILED=true
DEBUG= DEBUG=true
#################### FONCTIONS ############################################ #################### FONCTIONS ############################################
BOLD='' BOLD='\e[1m'
RED='' RED='\e[0;31m'
GREEN='' GREEN='\e[0;32m'
YELLOW='' YELLOW='\e[0;33m'
BLUE='' BLUE='\e[0;34m'
MAGENTA='' MAGENTA='\e[0;35m'
CYAN='' CYAN='\e[0;36m'
NC='' # No Color NC='\e[0m' # No Color
NL=' NL='
' '
#--------------------- Fichier de LOG ------------------- #--------------------- Fichier de LOG -------------------
LOG_FIC () { LOG_FIC () {
echo "${BLUE}$(date +%d-%m-%Y-%H-%M-%S)${NC} : $*" >> "${FIC_LOG}" echo -e "${BLUE}$(date +%d-%m-%Y-%H-%M-%S)${NC} : $*" >> "${TMP_LOG}"
} }
quitFilter () { quitFilter () {
LOG_FIC "${GREEN}######################################## filter stop${NC}" LOG_FIC "${GREEN}######################################## filter stop${NC}"
cat "${TMP_LOG}" >> "${FIC_LOG}"
rm -f "${TMP_LOG}"
exit $1 exit $1
} }
keepFailed () { keepFailed () {
[ -z "${KEEP_FAILED}" ] && return [ -z "${KEEP_FAILED}" ] && return
mkdir -p "${DIR_LOG}/pb/"
cp "$1" "${DIR_LOG}/pb/" cp "$1" "${DIR_LOG}/pb/"
} }
########################################
# curl Jirafeau
curlJirafeauUpdate () {
# $1: periode
# $2: jirafeauItemRef
LOG_FIC " - ${CYAN}curl -X POST -d \"u=$1\" -d \"h=$2\" \"${JIRAFEAU_LOCAL}/a.php}\""
curl -X POST -d "u=$1" -d "h=$2" "${JIRAFEAU_LOCAL}/a.php"
}
curlJirafeauSend () {
# $1: periode
# $2: filename
# $3: content-type
# $4: name
# $5: password
type="type=$3;"
[ -z "$3" -o "$3" = "/" ] && type=""
LOG_FIC " - curl -X POST -F \"time=$1\" -F \"key=$5\" -F \"file=@$2;${type}filename=\\\"$4\\\"\" \"${JIRAFEAU_LOCAL}/a.php\""
for num in {1..2}; do
OUTPUT=$(curl -X POST -F "time=$1" -F "key=$5" -F "file=@$2;${type}filename=\"$4\"" "${JIRAFEAU_LOCAL}/a.php")
read JIR_TOKEN <<< "${OUTPUT}"
case "${JIR_TOKEN}" in
"" | no | *Error* | \<* )
sleep 30
continue
;;
esac
break
done
echo "${OUTPUT}"
}
# Définir une fonction pour vérifier si le domaine d'un email est dans la liste SKIP_DOMAINS
function check_skip_domains() {
local SKIP_DOMAINS="$1"
local LIST_EMAILS="$2"
for email in $LIST_EMAILS; do
# Extraire le domaine de l'email (partie après le "@")
local domain="${email##*@}"
# Utiliser grep pour vérifier si le domaine est dans la liste des domaines à sauter
if echo "${SKIP_DOMAINS}" | grep -q -w "${domain}"; then
echo "yes"
return
fi
done
# Si aucun domaine n'a été trouvé, retourner "no"
echo "no"
}
#################### MAIN ################################################# #################### MAIN #################################################
echo "${NL}" >> "${FIC_LOG}" echo -e "${NL}${BLUE}$(date +%d-%m-%Y-%H-%M-%S)${NC} : ${GREEN}######################################## filter start (log in ${TMP_LOG})${NC}" >> "${FIC_LOG}"
LOG_FIC "${GREEN}######################################## filter start${NC}" LOG_FIC "${GREEN}######################################## ${TMP_LOG} ${NC}"
if ! mkdir -p "${MAILS}"; then if ! mkdir -p "${MAILS}"; then
LOG_FIC "${RED}Can't mkdir ${MAILS} ${NC}" LOG_FIC "${RED}Can't mkdir ${MAILS} ${NC}"
quitFilter "${EX_UNAVAILABLE}" quitFilter "${EX_UNAVAILABLE}"
fi fi
#$@ contient le sender et les destinataires
#format "-f sender -- dest1 [...dest(i)] "
LIST_EMAILS=$(echo $@)
#on nettoie
LIST_EMAILS=$(sed 's/-f//g' <<< $LIST_EMAILS)
LIST_EMAILS=$(sed 's/--//g' <<< $LIST_EMAILS)
MAIL_SOURCE=$(echo $@ | awk 'BEGIN{FS=" "} {print $2}') MAIL_SOURCE=$(echo $@ | awk 'BEGIN{FS=" "} {print $2}')
DATE_TEMPS=$(date "+%Y-%m-%d-%H:%M:%S") DATE_TEMPS=$(date "+%Y-%m-%d-%H:%M:%S")
REP_PIECE_JOINTE="${MAILS}/${DATE_TEMPS}_${MAIL_SOURCE}_$$" REP_PIECE_JOINTE="${MAILS}/${DATE_TEMPS}_${MAIL_SOURCE}_$$"
TRACK=$(curl "${JIRAFEAU_LOCAL}/a.php?r=${MAIL_SOURCE}" 2>/dev/null)
PERIOD=$(curl "${JIRAFEAU_LOCAL}/a.php?p=${MAIL_SOURCE}" 2>/dev/null)
if [ -n "$(echo "${PERIOD}" | grep -e minute -e hour -e day -e week -e month -e quarter 2>/dev/null)" ]; then
JIRAFEAU_TIME="${PERIOD}"
fi
LOG_FIC "\n" \ MODE=$(curl "${JIRAFEAU_LOCAL}/a.php?m=${MAIL_SOURCE}" 2>/dev/null )
" MAIL_SOURCE : ${YELLOW}${MAIL_SOURCE}${NC}\n" \ [[ "${MODE}" =~ ^(none|footer|attachment|both)$ ]] || MODE="${DEFAULT_MODE}"
" DATE_TEMPS : ${YELLOW}${DATE_TEMPS=}${NC}\n" \ TRACK=$(curl "${JIRAFEAU_LOCAL}/a.php?r=${MAIL_SOURCE}" 2>/dev/null )
" TRACK : ${YELLOW}${TRACK}${NC}\n" \ [[ "${TRACK}" =~ ^(|0|1|false|true|FALSE|TRUE|on|off)$ ]] || TRACK="${DEFAULT_TRACK}"
" PERIOD : ${YELLOW}${PERIOD}${NC}\n" \ PERIOD=$(curl "${JIRAFEAU_LOCAL}/a.php?p=${MAIL_SOURCE}" 2>/dev/null )
" JIRAFEAU_TIME: ${YELLOW}${JIRAFEAU_TIME}${NC}" [[ "${PERIOD}" =~ ^(minute|hour|day|week|month|quarter)$ ]] || PERIOD="${DEFAULT_PERIOD}"
LOG_FIC "${NL}" \
" MAIL_SOURCE : ${YELLOW}${MAIL_SOURCE}${NC}${NL}" \
" DATE_TEMPS : ${YELLOW}${DATE_TEMPS=}${NC}${NL}" \
" MODE : ${YELLOW}${MODE}${NC}${NL}" \
" TRACK : ${YELLOW}${TRACK}${NC}${NL}" \
" PERIOD : ${YELLOW}${PERIOD}${NC}${NL}"
if ! cd "${INSPECT_DIR}"; then if ! cd "${INSPECT_DIR}"; then
echo "${INSPECT_DIR} does not exist" echo "${INSPECT_DIR} does not exist"
@ -135,41 +208,55 @@ ARCHIVE_CONTENT="${REP_PIECE_JOINTE}/archive-content.txt"
JIRAFEAU_ERROR="${REP_PIECE_JOINTE}/jirafeau-error.txt" JIRAFEAU_ERROR="${REP_PIECE_JOINTE}/jirafeau-error.txt"
# Clean up when done or when aborting. # Clean up when done or when aborting.
[ -z "${DEBUG}" ] && trap "rm -rf in.$$ in.$$.altered ${REP_PIECE_JOINTE}" 0 1 2 3 15 [ -z "${DEBUG}" ] && trap "cd ${INSPECT_DIR}; rm -rf in.$$ in.$$.altered ${REP_PIECE_JOINTE}" 0 1 2 3 15
if ! cat > "in.$$"; then if ! cat > "${INSPECT_DIR}/in.$$"; then
LOG_FIC "${RED}Cannot save mail to file${NC}" LOG_FIC "${RED}Cannot save mail to file${NC}"
quitFilter "${EX_TEMPFAIL}" quitFilter "${EX_TEMPFAIL}"
fi fi
LOG_FIC "\n" \ dos2unix "${INSPECT_DIR}/in.$$" 2> /dev/null
LOG_FIC "${NL}" \
" size: ${YELLOW}$(wc -c < "${INSPECT_DIR}/in.$$")${NC}" " size: ${YELLOW}$(wc -c < "${INSPECT_DIR}/in.$$")${NC}"
[ -n "${DEBUG}" ] && (mkdir -p "${DIR_LOG}/pb/" ; cp "${INSPECT_DIR}/in.$$" "${DIR_LOG}/pb/in.$$.orig") [ -z "${DEBUG}" ] || (cp "${INSPECT_DIR}/in.$$" "${DIR_LOG}/pb/in.$$.orig")
mkdir -p "${REP_PIECE_JOINTE}/" mkdir -p "${REP_PIECE_JOINTE}/"
>"${OLD_LINKS}" >"${OLD_LINKS}"
>"${ARCHIVE_CONTENT}" >"${ARCHIVE_CONTENT}"
if [ "${MODE}" = "none" ]; then
LOG_FIC " - ${GREEN}send without change (MODE=none)${NC}"
${SENDMAIL} "$@" < "${INSPECT_DIR}/in.$$"
quitFilter 0
fi
if [ "$(check_skip_domains "$SKIP_DOMAINS" "$LIST_EMAILS")" = "yes" ]; then
LOG_FIC " - ${GREEN}send without change (skip domain)${NC}"
${SENDMAIL} "$@" < "${INSPECT_DIR}/in.$$"
quitFilter 0
fi
# Etape de rafraichissement des anciens fichiers inclus # Etape de rafraichissement des anciens fichiers inclus
echo "time: ${DATE_TEMPS}\nid: $(date +%s)" > "${ARCHIVE_CONTENT}" echo "time: ${DATE_TEMPS}${NL}id: $(date +%s)" > "${ARCHIVE_CONTENT}"
[ -n "${TRACK}" ] && echo "sender: ${MAIL_SOURCE}" >> "${ARCHIVE_CONTENT}" [ -n "${TRACK}" ] && echo "sender: ${MAIL_SOURCE}" >> "${ARCHIVE_CONTENT}"
LOG_FIC "${CYAN}${SHRINK_CMD} -u \"${INSPECT_DIR}/in.$$\" 2>> \"${FIC_LOG}\" > \"${OLD_LINKS}\"${NC}" LOG_FIC "${CYAN}${SHRINK_CMD} -u \"${INSPECT_DIR}/in.$$\" 2>> \"${TMP_LOG}\" > \"${OLD_LINKS}\"${NC}"
"${SHRINK_CMD}" -u "${INSPECT_DIR}/in.$$" 2>> "${FIC_LOG}" > "${OLD_LINKS}" "${SHRINK_CMD}" -u "${INSPECT_DIR}/in.$$" 2>> "${TMP_LOG}" > "${OLD_LINKS}"
cat "${OLD_LINKS}" | grep "${JIRAFEAU_URL}" | while read REMOTE_LINK; do cat "${OLD_LINKS}" | grep "${JIRAFEAU_URL}" | while read REMOTE_LINK; do
REMOTE_REF=$(echo "${REMOTE_LINK}" | sed -e 's/.*h=\([^&]*\).*/\1/' -e 's/.*http.*//') REMOTE_REF=$(echo "${REMOTE_LINK}" | sed -e 's/.*h=\([^&]*\).*/\1/' -e 's/.*http.*//')
[ -z "${REMOTE_REF}" ] && continue [ -z "${REMOTE_REF}" ] && continue
REMOTE_KEY=$(echo "${REMOTE_LINK}" | grep "k=" | sed 's%.*k=\([^&]*\).*%\1%') REMOTE_KEY=$(echo "${REMOTE_LINK}" | grep "k=" | sed 's%.*k=\([^&]*\).*%\1%')
# update periode for download # update periode for download
LOG_FIC " - ${CYAN}\"${JIRAFEAU_CMD}\" -f \"${JIRAFEAU_LOCAL}\" -t \"${JIRAFEAU_TIME}\" update \"${REMOTE_REF}\" 2>&1 >> \"${FIC_LOG}\"${NC}" curlJirafeauUpdate "${PERIOD}" "${REMOTE_REF}" 2>&1 >> "${TMP_LOG}"
"${JIRAFEAU_CMD}" -f "${JIRAFEAU_LOCAL}" -t "${JIRAFEAU_TIME}" update "${REMOTE_REF}" 2>&1 >> "${FIC_LOG}"
echo "old: ${REMOTE_REF} ${REMOTE_KEY}" >> "${ARCHIVE_CONTENT}" echo "old: ${REMOTE_REF} ${REMOTE_KEY}" >> "${ARCHIVE_CONTENT}"
done done
LOG_FIC " - archive starts with: ${NL}${YELLOW}$(cat ${ARCHIVE_CONTENT})${NC}" LOG_FIC " - archive starts with: ${NL}${YELLOW}$(cat ${ARCHIVE_CONTENT})${NC}"
# Etape extraction des pieces jointes # Etape extraction des pieces jointes
LOG_FIC "${CYAN}${SHRINK_CMD} -s ${MAX_KEEP_IN_MAIL} -d ${REP_PIECE_JOINTE} ${INSPECT_DIR}/in.$$ ${NC}" LOG_FIC "${CYAN}${SHRINK_CMD} -s \"${MAX_KEEP_IN_MAIL}\" -d \"${REP_PIECE_JOINTE}\" \"${INSPECT_DIR}/in.$$\"${NC}"
"${SHRINK_CMD}" -s "${MAX_KEEP_IN_MAIL}" -d "${REP_PIECE_JOINTE}" "${INSPECT_DIR}/in.$$" 2>> "${FIC_LOG}" | { "${SHRINK_CMD}" -s "${MAX_KEEP_IN_MAIL}" -d "${REP_PIECE_JOINTE}" "${INSPECT_DIR}/in.$$" 2>> "${TMP_LOG}" | {
while read ATTACH_TMP_NAME; do while read ATTACH_TMP_NAME; do
if [ -d "${ATTACH_TMP_NAME}" ]; then if [ -d "${ATTACH_TMP_NAME}" ]; then
ATTACH_MEDIA="${ATTACH_TMP_NAME}/media" ATTACH_MEDIA="${ATTACH_TMP_NAME}/media"
@ -181,10 +268,9 @@ LOG_FIC "${CYAN}${SHRINK_CMD} -s ${MAX_KEEP_IN_MAIL} -d ${REP_PIECE_JOINTE} ${IN
continue continue
fi fi
# Etape de televersement des pieces jointes # Etape de televersement des pieces jointes
PASSWORD=$(apg -n 1 -m 12) PASSWORD=$(apg -n 1 -m 12 -M cln)
PASSWORD_MD5=$(echo -n ${PASSWORD} | ${MD5_CMD} | cut -d \ -f 1) PASSWORD_MD5=$(echo -n ${PASSWORD} | ${MD5_CMD} | cut -d \ -f 1)
LOG_FIC " - ${CYAN}\"${JIRAFEAU_CMD}\" -f \"${JIRAFEAU_LOCAL}\" -t \"${JIRAFEAU_TIME}\" -s \"${MAX_UPLOAD_SIZE}\" -c \"${ATTACH_CONTENT_TYPE}\" -n \"${ATTACH_NAME}\" send \"${ATTACH_MEDIA}\" \"${PASSWORD}\" 2>> \"${FIC_LOG}\" > \"${ONE_LINK}\"${NC}" curlJirafeauSend "${PERIOD}" "${ATTACH_MEDIA}" "${ATTACH_CONTENT_TYPE}" "${ATTACH_NAME}" "${PASSWORD}" 2>> "${TMP_LOG}" > "${ONE_LINK}"
"${JIRAFEAU_CMD}" -f "${JIRAFEAU_LOCAL}" -t "${JIRAFEAU_TIME}" -s "${MAX_UPLOAD_SIZE}" -c "${ATTACH_CONTENT_TYPE}" -n "${ATTACH_NAME}" send "${ATTACH_MEDIA}" "${PASSWORD}" 2>> "${FIC_LOG}" > "${ONE_LINK}"
cat "${ONE_LINK}" | { cat "${ONE_LINK}" | {
read JIR_TOKEN read JIR_TOKEN
read JIR_CODE read JIR_CODE
@ -197,8 +283,8 @@ LOG_FIC "${CYAN}${SHRINK_CMD} -s ${MAX_KEEP_IN_MAIL} -d ${REP_PIECE_JOINTE} ${IN
echo "UPLOAD_FAIL" >> "${JIRAFEAU_ERROR}" echo "UPLOAD_FAIL" >> "${JIRAFEAU_ERROR}"
;; ;;
* ) * )
LOG_FIC " - change by link ${YELLOW}${JIRAFEAU_URL}/f.php?d=1&h=${JIR_TOKEN}&k=${PASSWORD_MD5}${NC}" LOG_FIC " - change by link ${YELLOW}${JIRAFEAU_URL}/f.php?d=0&h=${JIR_TOKEN}&k=${PASSWORD_MD5}${NC}"
echo "url: ${JIRAFEAU_URL}/f.php?d=1&h=${JIR_TOKEN}&k=${PASSWORD_MD5}" echo "url: ${JIRAFEAU_URL}/f.php?d=0&h=${JIR_TOKEN}&k=${PASSWORD_MD5}"
echo "new: ${JIR_TOKEN} ${PASSWORD_MD5}" >> "${ARCHIVE_CONTENT}" echo "new: ${JIR_TOKEN} ${PASSWORD_MD5}" >> "${ARCHIVE_CONTENT}"
;; ;;
esac esac
@ -207,11 +293,11 @@ LOG_FIC "${CYAN}${SHRINK_CMD} -s ${MAX_KEEP_IN_MAIL} -d ${REP_PIECE_JOINTE} ${IN
# Création de l'archive # Création de l'archive
NB_ATTACH=$(grep -e "^old: " -e "^new: " "${ARCHIVE_CONTENT}" | wc -l) NB_ATTACH=$(grep -e "^old: " -e "^new: " "${ARCHIVE_CONTENT}" | wc -l)
if [ \( -n "${TRACK}" -a "${NB_ATTACH}" -gt 0 \) -o "${NB_ATTACH}" -gt 1 ]; then if [ \( -n "${TRACK}" -a "${NB_ATTACH}" -gt 0 \) -o "${NB_ATTACH}" -gt 1 ]; then
PASSWORD=$(apg -n 1 -m 12) PASSWORD=$(apg -n 1 -m 12 -M cln)
PASSWORD_MD5=$(echo -n ${PASSWORD} | ${MD5_CMD} | cut -d \ -f 1) PASSWORD_MD5=$(echo -n ${PASSWORD} | ${MD5_CMD} | cut -d \ -f 1)
LOG_FIC " - ${MAGENTA}upload archive${NC}" LOG_FIC " - ${MAGENTA}upload archive${NC}"
LOG_FIC " - ${CYAN}\"${JIRAFEAU_CMD}\" -f \"${JIRAFEAU_LOCAL}\" -t \"${JIRAFEAU_TIME}\" -s \"${MAX_UPLOAD_SIZE}\" -c \"${ARCHIVE_MIME}\" -n \"${ARCHIVE_TITLE}\" send \"${ARCHIVE_CONTENT}\" \"${PASSWORD}\" 2>> \"${FIC_LOG}\" > \"${ONE_LINK}\"${NC}"
"${JIRAFEAU_CMD}" -f "${JIRAFEAU_LOCAL}" -t "${JIRAFEAU_TIME}" -s "${MAX_UPLOAD_SIZE}" -c "${ARCHIVE_MIME}" -n "${ARCHIVE_TITLE}" send "${ARCHIVE_CONTENT}" "${PASSWORD}" 2>> "${FIC_LOG}" > "${ONE_LINK}" curlJirafeauSend "${PERIOD}" "${ARCHIVE_CONTENT}" "${ARCHIVE_MIME}" "${ARCHIVE_TITLE}" "${PASSWORD}" 2>> "${TMP_LOG}" > "${ONE_LINK}"
fi fi
LOG_FIC " - final archive content: ${NL}${YELLOW}$(cat ${ARCHIVE_CONTENT})${NC}" LOG_FIC " - final archive content: ${NL}${YELLOW}$(cat ${ARCHIVE_CONTENT})${NC}"
if [ "${NB_ATTACH}" -gt 1 ]; then if [ "${NB_ATTACH}" -gt 1 ]; then
@ -237,10 +323,10 @@ LOG_FIC "${CYAN}${SHRINK_CMD} -s ${MAX_KEEP_IN_MAIL} -d ${REP_PIECE_JOINTE} ${IN
echo "arch: none" echo "arch: none"
fi fi
# Etape de substitution # Etape de substitution
LOG_FIC "${CYAN}${SHRINK_CMD} -s \"${MAX_KEEP_IN_MAIL}\" \"${INSPECT_DIR}/in.$$\" \"${INSPECT_DIR}/in.$$.altered\" 2>> \"${FIC_LOG}\"${NC}" LOG_FIC "${CYAN}${SHRINK_CMD} -m \"${MODE}\" -s \"${MAX_KEEP_IN_MAIL}\" \"${INSPECT_DIR}/in.$$\" \"${INSPECT_DIR}/in.$$.altered\" 2>> \"${TMP_LOG}\"${NC}"
} | "${SHRINK_CMD}" -s "${MAX_KEEP_IN_MAIL}" "${INSPECT_DIR}/in.$$" "${INSPECT_DIR}/in.$$.altered" 2>> "${FIC_LOG}" } | "${SHRINK_CMD}" -m "${MODE}" -s "${MAX_KEEP_IN_MAIL}" "${INSPECT_DIR}/in.$$" "${INSPECT_DIR}/in.$$.altered" 2>> "${TMP_LOG}"
[ -n "${DEBUG}" ] && (mkdir -p "${DIR_LOG}/pb/" ; cp "${INSPECT_DIR}/in.$$.altered" "${DIR_LOG}/pb/in.$$.altered") [ -z "${DEBUG}" ] || (cp "${INSPECT_DIR}/in.$$.altered" "${DIR_LOG}/pb/in.$$.altered")
if [ -s "${JIRAFEAU_ERROR}" ]; then if [ -s "${JIRAFEAU_ERROR}" ]; then
LOG_FIC " - ${RED}upload fail${NC}" LOG_FIC " - ${RED}upload fail${NC}"
@ -251,7 +337,7 @@ fi
if [ "$(wc -l < "${ARCHIVE_CONTENT}")" -ge 3 ]; then if [ "$(wc -l < "${ARCHIVE_CONTENT}")" -ge 3 ]; then
# verification de taille finale # verification de taille finale
actualSize=$(wc -c < "${INSPECT_DIR}/in.$$.altered") actualSize=$(wc -c < "${INSPECT_DIR}/in.$$.altered")
if [ ${actualSize} -ge $MAX_FINAL_SIZE ]; then if [ "${actualSize}" -ge "${MAX_FINAL_SIZE}" ]; then
LOG_FIC " - ${RED}too big even after diet ${INSPECT_DIR}/in.$$.altered (${actualSize})${NC}" LOG_FIC " - ${RED}too big even after diet ${INSPECT_DIR}/in.$$.altered (${actualSize})${NC}"
keepFailed "${INSPECT_DIR}/in.$$" keepFailed "${INSPECT_DIR}/in.$$"
quitFilter "${EX_TOO_LARGE}" quitFilter "${EX_TOO_LARGE}"
@ -261,15 +347,16 @@ if [ "$(wc -l < "${ARCHIVE_CONTENT}")" -ge 3 ]; then
else else
# verification de taille finale # verification de taille finale
actualSize=$(wc -c < "${INSPECT_DIR}/in.$$") actualSize=$(wc -c < "${INSPECT_DIR}/in.$$")
if [ ${actualSize} -ge $MAX_FINAL_SIZE ]; then if [ "${actualSize}" -ge "${MAX_FINAL_SIZE}" ]; then
LOG_FIC " - ${RED}too big without diet ${INSPECT_DIR}/in.$$ (${actualSize}) ${NC}" LOG_FIC " - ${RED}too big without diet ${INSPECT_DIR}/in.$$ (${actualSize}) ${NC}"
keepFailed "${INSPECT_DIR}/in.$$" keepFailed "${INSPECT_DIR}/in.$$"
quitFilter "${EX_TOO_LARGE}" quitFilter "${EX_TOO_LARGE}"
fi fi
LOG_FIC " - ${GREEN}send without attach file${NC}" LOG_FIC " - ${GREEN}send without attach file${NC}"
${SENDMAIL} "$@" < "in.$$" ${SENDMAIL} "$@" < "${INSPECT_DIR}/in.$$"
fi fi
quitFilter 0 quitFilter 0
########################################################################## ##########################################################################

233
src/bash/filterTest.sh Executable file
View File

@ -0,0 +1,233 @@
#!/bin/bash
##########################################################################
# Copyright KAZ 2021 #
# #
# contact (at) kaz.bzh #
# #
# This software is a filter to shrink email by attachment extraction. #
# #
# This software is governed by the CeCILL-B license under French law and #
# abiding by the rules of distribution of free software. You can use, #
# modify and/or redistribute the software under the terms of the #
# CeCILL-B license as circulated by CEA, CNRS and INRIA at the following #
# URL "http://www.cecill.info". #
# #
# As a counterpart to the access to the source code and rights to copy, #
# modify and redistribute granted by the license, users are provided #
# only with a limited warranty and the software's author, the holder of #
# the economic rights, and the successive licensors have only limited #
# liability. #
# #
# In this respect, the user's attention is drawn to the risks associated #
# with loading, using, modifying and/or developing or reproducing the #
# software by the user in light of its specific status of free software, #
# that may mean that it is complicated to manipulate, and that also #
# therefore means that it is reserved for developers and experienced #
# professionals having in-depth computer knowledge. Users are therefore #
# encouraged to load and test the software's suitability as regards #
# their requirements in conditions enabling the security of their #
# systems and/or data to be ensured and, more generally, to use and #
# operate it in the same conditions as regards security. #
# #
# The fact that you are presently reading this means that you have had #
# knowledge of the CeCILL-B license and that you accept its terms. #
##########################################################################
PRG=$(basename $0)
ATTACH_MODE="FOOTER"
export SIMULATE=""
BOLD='\e[1m'
RED='\e[0;31m'
GREEN='\e[0;32m'
YELLOW='\e[0;33m'
BLUE='\e[0;34m'
MAGENTA='\e[0;35m'
CYAN='\e[0;36m'
NC='\e[0m' # No Color
NL='
'
########################################
LOG () {
echo -e "$1" 1>&2
}
usage () {
echo "Usage: ${PRG} [-h|-v|-g] [-m {NONE|FOOTER|ATTACHMENT|BOTH}] mbox"
exit 1
}
while : ; do
case "$1" in
-h*) usage;;
-v*) "${eMailShrinker}" -v; exit;;
-g) DEBUG="-g"; shift;;
-s) SIMULATE="echo"; shift;;
-m) shift; ATTACH_MODE="$1"; shift;;
*) break;;
esac
done
case "${ATTACH_MODE}" in
""|NONE|FOOTER|ATTACHMENT|BOTH);;
*) usage;;
esac
[ -z "${ATTACH_MODE}" ] || ATTACH_MODE="-m ${ATTACH_MODE}"
[ "$#" -eq 1 ] || usage
mbox=$(realpath "$1")
########################################
# recherche des binaires
cd $(dirname $0)
eMailShrinker="$(realpath "./eMailShrinker")"
[ -x "${eMailShrinker}" ] || eMailShrinker="$(realpath "../../build/out/eMailShrinker")"
[ -x "${eMailShrinker}" ] || ( echo -e "${RED}eMailShrinker not found${NC}" ; exit)
########################################
dos2unix "${mbox}"
[ -z "${SIMULATE}" ] && DOMAINEDEPOT="$(cat domainedepot)"
JIRAFEAU_URL="https://depot.${DOMAINEDEPOT}"
JIRAFEAU_LOCAL="${JIRAFEAU_URL}"
TMP_DIR="$(mktemp)"
########################################
# curl Jirafeau
curlJirafeauUpdate () {
# $1: periode
# $2: jirafeauItemRef
[ -z "${SIMULATE}" ] && curl -X POST -d "u=$1" -d "h=$2" "${JIRAFEAU_LOCAL}/a.php"
}
curlJirafeauSend () {
# $1: periode
# $2: filename
# $3: content-type
# $4: name
# $5: password
if [ -n "${SIMULATE}" ]; then
echo -e "TokenXXX\nCodeX"
return
fi
type="type=$3;"
[ -z "$3" -o "$3" = "/" ] && type=""
LOG "curl -X POST -F \"time=$1\" -F \"key=$5\" -F \"file=@$2;${type}filename=\\\"$4\\\"\" \"${JIRAFEAU_LOCAL}/a.php\""
curl -X POST -F "time=$1" -F "key=$5" -F "file=@$2;${type}filename=\"$4\"" "${JIRAFEAU_LOCAL}/a.php" || exit 1
}
########################################
# nettoyage
rm -f "${TMP_DIR}" ; mkdir -p "${TMP_DIR}"
rm -fr "${TMP_DIR}/PJ-name.txt" "${TMP_DIR}/PJ-Keys.txt" "${TMP_DIR}/PJ" "${TMP_DIR}/archive-content.txt" "${TMP_DIR}/url-to-refresh.txt" "${TMP_DIR}/new-mbox"
echo -e "time: $(date "+%Y-%m-%d-%H:%M:%S")\nid: $(date +%s)" > "${TMP_DIR}/archive-content.txt"
########################################
# affichage de la structure de départ
LOG " - ${BLUE}mbox: ${mbox}${NC}${YELLOW}"
"${eMailShrinker}" -l "${mbox}"
LOG "${NC}"
########################################
# recherche des prolongations des délais de grace
"${eMailShrinker}" ${DEBUG} -u "${mbox}" > "${TMP_DIR}/url-to-refresh.txt"
cat "${TMP_DIR}/url-to-refresh.txt" | grep "${JIRAFEAU_URL}" | while read REMOTE_LINK; do
REMOTE_REF=$(echo "${REMOTE_LINK}" | sed -e 's/.*h=\([^&]*\).*/\1/' -e 's/.*http.*//')
[ -z "${REMOTE_REF}" ] && continue
LOG " - ${BLUE}update ${REMOTE_REF}${NC}"
curlJirafeauUpdate "month" "${REMOTE_REF}"
echo "old: ${REMOTE_REF} ${REMOTE_KEY}" >> "${TMP_DIR}/archive-content.txt"
done
########################################
# extraction des pièces jointes
"${eMailShrinker}" ${DEBUG} -s "5ki" -d "${TMP_DIR}/PJ" "${mbox}" > "${TMP_DIR}/PJ-name.txt" || exit 1
LOG " - ${BLUE}PJ-name: ${NC}"
cat "${TMP_DIR}/PJ-name.txt"
LOG
########################################
# dépot des extractions dans jirafeau et récupération des codes
cat "${TMP_DIR}/PJ-name.txt" | {
while read ATTACH_TMP_NAME; do
LOG " - ${BLUE}find ATTACH_TMP_NAME: (${ATTACH_TMP_NAME}) ${NC}"
if [ -d "${ATTACH_TMP_NAME}" ]; then
ATTACH_MEDIA="${ATTACH_TMP_NAME}/media"
ATTACH_NAME=$(grep "^Name: " "${ATTACH_TMP_NAME}/meta" | cut -c 7- )
ATTACH_CONTENT_TYPE=$(grep "^Content-Type: " "${ATTACH_TMP_NAME}/meta" | cut -c 15- )
else
LOG " - ${RED}no ATTACH_TMP_NAME: ${ATTACH_TMP_NAME}${NC}"
# XXX error
continue
fi
LOG " - ${BLUE}find ${ATTACH_NAME} / (${ATTACH_CONTENT_TYPE}) / ${ATTACH_MEDIA} ${NC}"
PASSWORD=$(apg -n 1 -m 12 -M cln)
PASSWORD_MD5=$(echo -n ${PASSWORD} | ${MD5_CMD} | cut -d \ -f 1)
curlJirafeauSend "month" "${ATTACH_MEDIA}" "${ATTACH_CONTENT_TYPE}" "${ATTACH_NAME}" "${PASSWORD}" > "${TMP_DIR}/one.txt"
cat "${TMP_DIR}/one.txt" | {
read JIR_TOKEN
read JIR_CODE
case "${JIR_TOKEN}" in
"" | no | *Error* | \<* )
LOG " - ${RED}can't upload ${ATTACH_MEDIA} <${JIR_TOKEN}> <${JIR_CODE}>${NC}"
cat "${TMP_DIR}/one.txt" >&2
echo "url:"
exit 1
;;
* )
LOG " - ${GREEN} upload ${ATTACH_MEDIA}${NC}"
echo "url: ${JIRAFEAU_URL}/f.php?d=1&h=${JIR_TOKEN}&k=${PASSWORD_MD5}"
echo "new: ${JIR_TOKEN} ${PASSWORD_MD5}" >> "${TMP_DIR}/archive-content.txt"
;;
esac
}
done
NB_ATTACH=$(grep -e "^old: " -e "^new: " "${TMP_DIR}/archive-content.txt" | wc -l)
if [ "${NB_ATTACH}" -gt 1 ]; then
PASSWORD=$(apg -n 1 -m 12 -M cln)
PASSWORD_MD5=$(echo -n ${PASSWORD} | ${MD5_CMD} | cut -d \ -f 1)
curlJirafeauSend "month" "${TMP_DIR}/archive-content.txt" "text/kaz_email_archive" "archive_content" "${PASSWORD}" > "${TMP_DIR}/one.txt" || exit 1
cat "${TMP_DIR}/one.txt" | {
read JIR_TOKEN
read JIR_CODE
case "${JIR_TOKEN}" in
"" | no | *Error* | \<* )
LOG " - ${RED}can't upload ${TMP_DIR}/archive-content.txt${NC}"
echo "arch: bad"
exit 1
;;
* )
LOG " - ${GREEN} upload archive-content.txt${NC}"
echo "arch: ${JIRAFEAU_URL}/a.php?g=${JIR_TOKEN}~${PASSWORD_MD5}"
;;
esac
}
else
LOG " - ${GREEN}no archive${NC}"
echo "arch: none"
fi
} > "${TMP_DIR}/PJ-Keys.txt"
LOG " - ${BLUE}PJ-Keys: ${NC}"
cat "${TMP_DIR}/PJ-Keys.txt"
LOG
LOG " - ${GREEN}ATTACH_MODE: ${ATTACH_MODE}${NC}"
########################################
# substitution des pièces jointes par les codes fournis par jirafeau
cat "${TMP_DIR}/PJ-Keys.txt" | "${eMailShrinker}" ${DEBUG} ${ATTACH_MODE} -s "5ki" "${mbox}" "${TMP_DIR}/new-mbox" || exit 1
########################################
# affichage de la structure à la fin
LOG " - ${BLUE}new-mbox:${NC}${YELLOW}"
"${eMailShrinker}" -l "${TMP_DIR}/new-mbox" || ( LOG "${NC}" ; exit 1)
echo -e "\n${NC}${GREEN}${BOLD}resul in ${TMP_DIR}/new-mbox${NC}"
exit 0

View File

@ -0,0 +1,8 @@
#!/bin/bash
TMP_LOG="$(mktemp)"
#trap "rm -rf ${TMP_LOG}" 0 1 2 3 15
cat > ${TMP_LOG}
cat ${TMP_LOG}

View File

@ -60,22 +60,17 @@ const string Attachment::contentIDToken ("content-id");
const string Attachment::PLAIN ("plain"); const string Attachment::PLAIN ("plain");
const string Attachment::HTML ("html"); const string Attachment::HTML ("html");
const string Attachment::RELATED ("related"); const string Attachment::RELATED ("related");
const string Attachment::SIGNED ("signed");
const string Attachment::ALTERNATIVE ("alternative"); const string Attachment::ALTERNATIVE ("alternative");
const string Attachment::KAZ_ATTACH_NAME ("vos-pieces-jointes-kaz-ici.htm");
const string Attachment::MULTIPART ("multipart/");
const regex Attachment::nameCharsetRegEx ( ".*name\\*=\\s*([; \t]*)");
const regex Attachment::nameCharsetRegEx (".*name\\*=(.*)"); const regex Attachment::nameRegEx ( ".*name=\\s*((\"(\\\\.|[^\\\\])*\")|[^; \t]*).*");
const regex Attachment::nameRegEx (".*name=\"([^\"]*)\".*"); const regex Attachment::boundaryRegEx (".*boundary=\\s*((\"(\\\\.|[^\\\\])*\")|[^; \t]*).*");
// boundary="----=_Part_796779_1154936629.1668080348646"
// boundary="------------040709000505010508040808"
// boundary="----------=_1668606031-941125-91"
// boundary="_004_PAVPR10MB6792713B313048E3A259B215B2079PAVPR10MB6792EURP_";
// boundary="_000_PAVPR10MB6792713B313048E3A259B215B2079PAVPR10MB6792EURP_"
// boundary=--boundary_1351_64006126-2b0e-4a3b-98ac-4797d1634188
// boundary=--boundary_1352_7e294c9a-cfab-44a0-bfb3-7310380ac7cb;
const regex Attachment::boundaryRegEx (".*boundary=\"?([^\"; ]*)\"?;?.*");
const regex Attachment::cidDefRegEx (".*<([^>]*)>.*"); const regex Attachment::cidDefRegEx (".*<([^>]*)>.*");
const regex Attachment::textRegEx (".*text/("+PLAIN+"|"+HTML+").*"); const regex Attachment::textRegEx (".*text/("+PLAIN+"|"+HTML+").*");
const regex Attachment::multiRegEx ("\\s*multipart/(mixed|"+RELATED+"|"+ALTERNATIVE+").*"); const regex Attachment::multiRegEx ("\\s*"+MULTIPART+"(mixed|"+RELATED+"|"+ALTERNATIVE+"|"+SIGNED+").*");
const string Attachment::IMG_BEGIN ("<IMG"); const string Attachment::IMG_BEGIN ("<IMG");
const string Attachment::IMG_END (">"); const string Attachment::IMG_END (">");
@ -130,7 +125,7 @@ Attachment::removeSection (string &content, const string &beginTag, const string
// ================================================================================ // ================================================================================
string string
Attachment::getSection (const string &content, const string &beginTag, const string &endTag) { Attachment::getSection (const string &content, const string &beginTag, const string &endTag) {
DEF_LOG ("Attachment::getSection", "beginTag: " << beginTag << " endTag: " << endTag << " content: " << content); DEF_LOG ("Attachment::getSection", "beginTag: " << beginTag << " endTag: " << endTag << " content: " << content.substr (0, 100) << "...");
vector<string> list; vector<string> list;
getSection (content, beginTag, endTag, list); getSection (content, beginTag, endTag, list);
size_t sum (0); size_t sum (0);
@ -147,14 +142,14 @@ Attachment::getSection (const string &content, const string &beginTag, const str
// ================================================================================ // ================================================================================
void void
Attachment::getSection (const string &content, const string &beginTag, const string &endTag, vector<string> &result) { Attachment::getSection (const string &content, const string &beginTag, const string &endTag, vector<string> &result) {
DEF_LOG ("Attachment::getSection", "beginTag: " << beginTag << " endTag: " << endTag << " content: " << content); DEF_LOG ("Attachment::getSection", "beginTag: " << beginTag << " endTag: " << endTag << " content: " << content.substr (0, 100) << "...");
for (string::size_type startPos (0); for (string::size_type startPos (0);
(startPos = caseInsensitiveFind (content, beginTag, startPos)) != string::npos; (startPos = caseInsensitiveFind (content, beginTag, startPos)) != string::npos;
) { ) {
LOG (beginTag << ": " << startPos); LOG (beginTag << ": " << startPos);
string::size_type stopPos = caseInsensitiveFind (content, endTag, startPos); string::size_type stopPos = caseInsensitiveFind (content, endTag, startPos);
LOG_BUG (stopPos == string::npos, break, "eMailShrinker: bug A3: " << endTag << " not found! at: " << startPos); LOG_BUG (stopPos == string::npos, break, "eMailShrinker: bug A3: " << endTag << " not found! at: " << startPos << endl << content.substr (0, 100) << "...");
LOG ("start: " << startPos << " stop: " << stopPos); LOG ("start: " << startPos << " stop: " << stopPos);
LOG_BUG (startPos == stopPos, /* */, "eMailShrinker: bug A4: " << endTag << " without " << beginTag << " at: " << startPos); LOG_BUG (startPos == stopPos, /* */, "eMailShrinker: bug A4: " << endTag << " without " << beginTag << " at: " << startPos);
@ -181,32 +176,51 @@ Attachment::getContentType () const {
const string const string
Attachment::getAttachName () const { Attachment::getAttachName () const {
static string tokens [] = {contentTypeToken, contentDispositionToken};
DEF_LOG ("Attachment::getAttachName", ""); DEF_LOG ("Attachment::getAttachName", "");
string result = getProp (contentTypeToken, nameRegEx); for (string token : tokens) {
// name=
string result = getProp (token, nameRegEx);
removeQuote (result);
if (result.length ()) { if (result.length ()) {
LOG ("name=: " << result); LOG ("name=: " << result);
encodedWord (result); encodedWordDecode (result);
return result; return result;
} }
result = getProp (contentTypeToken, nameCharsetRegEx); // name*x=
for (int id = 0; ; ++id) {
string item = getProp (token, regex (".*name\\*"+to_string (id)+"=\\s*((\"(\\\\.|[^\\\\])*\")|[; \t]*).*"));
if (item.empty ())
break;
result += item;
}
removeQuote (result);
if (result.length ()) {
LOG ("name*x=: " << result);
encodedWordDecode (result);
return result;
}
// name*=
result = getProp (token, nameCharsetRegEx);
removeQuote (result);
if (result.length ()) { if (result.length ()) {
LOG ("name*=: " << result); LOG ("name*=: " << result);
charsetValue (result); charsetValueDecode (result);
return result; return result;
} }
// XXX il faut composer s'il y a plusieurs ligne filename*x= // name*x*=
result = getProp (contentDispositionToken, nameRegEx); for (int id = 0; ; ++id) {
string item = getProp (token, regex (".*name\\*"+to_string (id)+"\\*=\\s*([^; ]*)"));
if (item.empty ())
break;
result += item;
}
removeQuote (result);
if (result.length ()) { if (result.length ()) {
LOG ("filename=: " << result); LOG ("name*x*=: " << result);
encodedWord (result); encodedWordDecode (result);
return result; return result;
} }
// XXX il faut composer s'il y a plusieurs ligne filename*x*=
result = getProp (contentDispositionToken, nameRegEx);
if (result.length ()) {
LOG ("filename*=: " << result);
charsetValue (result);
return result;
} }
return getUnknown (getContentType ()); return getUnknown (getContentType ());
} }
@ -260,7 +274,7 @@ Attachment::isDefProp (const string &token, const string &val) const {
if (it == env.end ()) if (it == env.end ())
return false; return false;
// XXX case insensitive ?? // XXX case insensitive ??
return it->second.find (val) != string::npos; return caseInsensitiveFind (it->second, val) != string::npos;
} }
// ================================================================================ // ================================================================================
@ -273,6 +287,8 @@ Attachment::Attachment (ifstream &mbox, const int &level, const streamoff beginI
toExtract (false), toExtract (false),
toUpdate (false), toUpdate (false),
toDisclaim (false), toDisclaim (false),
isKazAttachment (false),
isSigned (false),
boundaryMiddleSize (0) { boundaryMiddleSize (0) {
DEF_LOG ("Attachment::Attachment", "curPos: " << curPos << " level: " << level); DEF_LOG ("Attachment::Attachment", "curPos: " << curPos << " level: " << level);
readMime (mbox, curPos); readMime (mbox, curPos);
@ -280,6 +296,38 @@ Attachment::Attachment (ifstream &mbox, const int &level, const streamoff beginI
} }
// ================================================================================ // ================================================================================
/*! lower case "var in string " VAR=val; ..." */
inline string
cleanString (string line) {
DEF_LOG ("Attachment::cleanString", "line: " << line.substr (0, 100) << "...");
static const regex findPart ("(((\"(\\\\.|[^\\\\])*\")|\\s|[^;\\\"])+;?)");
static const regex findVarVal ("(\\s*[a-zA-Z_*-]+(\\*[0-9]+\\*)?=)(.*;?)");
string result;
smatch m1, m2;
while (regex_search (line, m1, findPart)) {
if (m1.prefix ().length ())
result += m1.prefix (); // XXX when \" on multi-lines
string part (m1[1]);
LOG ("part: " << part);
if (regex_search (part, m2, findVarVal) && !m2.prefix ().length ()) {
string id (m2[1]);
LOG ("id: " << id);
toLower (id);
result += id;
result += m2[3];
} else
result += part;
line = m1.suffix ();
}
if (!line.empty () && line[line.size() - 1] == '\r')
line = line.substr (0, line.size () - 1);
result += line;
LOG ("result: " << result);
return result;
}
void void
Attachment::readMime (ifstream &mbox, streamoff &curPos) { Attachment::readMime (ifstream &mbox, streamoff &curPos) {
DEF_LOG ("Attachment::readMime", "curPos: " << curPos); DEF_LOG ("Attachment::readMime", "curPos: " << curPos);
@ -288,7 +336,7 @@ Attachment::readMime (ifstream &mbox, streamoff &curPos) {
for (; getline (mbox, line); ) { for (; getline (mbox, line); ) {
LOG ("pos: " << curPos << " line: " << line); LOG ("pos: " << curPos << " line: " << line);
curPos += line.length () + 1; curPos += line.length () + 1;
if (line.empty ()) if (line.empty () || "\r" == line)
break; break;
if (line[0] == ' ' || line[0] == '\t') { if (line[0] == ' ' || line[0] == '\t') {
if (lastVar.empty ()) { if (lastVar.empty ()) {
@ -296,8 +344,8 @@ Attachment::readMime (ifstream &mbox, streamoff &curPos) {
LOG_BUG (true, /* */, "eMailShrinker: bug A5: not compliant MIME. pos: " << (curPos - (line.length () + 1)) << " line: " << line); LOG_BUG (true, /* */, "eMailShrinker: bug A5: not compliant MIME. pos: " << (curPos - (line.length () + 1)) << " line: " << line);
} else { } else {
LOG ("add line to var: " << line); LOG ("add line to var: " << line);
env.find (lastVar)->second += line; env.find (lastVar)->second += cleanString (line);
LOG ("new val: " << env.find (lastVar)->second); LOG ("new val(a): <" << lastVar << " <=> " << env.find (lastVar)->second << ">");
} }
continue; continue;
} }
@ -306,8 +354,11 @@ Attachment::readMime (ifstream &mbox, streamoff &curPos) {
lastVar = line.substr (0, colonPos); lastVar = line.substr (0, colonPos);
toLower (lastVar); toLower (lastVar);
LOG ("find var: " << lastVar); LOG ("find var: " << lastVar);
string val (line.length () >= colonPos+2 ? line.substr (colonPos+2) : ""); // XXX check RFC " " after ": " ++colonPos;
LOG ("new var: " << lastVar << " <=> " << val); if (line.length () >= colonPos && line [colonPos] == ' ')
++colonPos;
string val (cleanString (line.length () >= colonPos ? line.substr (colonPos) : ""));
LOG ("new var(b): <" << lastVar << " <=> " << val << ">");
env [lastVar] = val; env [lastVar] = val;
} }
} }
@ -315,12 +366,19 @@ Attachment::readMime (ifstream &mbox, streamoff &curPos) {
contentPos = curPos; contentPos = curPos;
cid = getProp (contentIDToken, cidDefRegEx); cid = getProp (contentIDToken, cidDefRegEx);
if (caseInsensitiveFind (getContentType (), MULTIPART) != string::npos) {
string multiProp = getProp (contentTypeToken, multiRegEx);
if (SIGNED == multiProp)
isSigned = true;
boundary = getProp (contentTypeToken, boundaryRegEx); boundary = getProp (contentTypeToken, boundaryRegEx);
removeQuote (boundary);
LOG ("boundary: " << boundary); LOG ("boundary: " << boundary);
if (boundary.length ()) { if (boundary.length ()) {
boundary = "--"+boundary+"--"; boundary = "--"+boundary+"--";
boundaryMiddleSize = boundary.length () - 2; boundaryMiddleSize = boundary.length () - 2;
} }
}
LOG ("readMime contentPos: " << contentPos << " cid: " << cid << " boundary: " << boundary); LOG ("readMime contentPos: " << contentPos << " cid: " << cid << " boundary: " << boundary);
} }
@ -408,15 +466,19 @@ Attachment::markDisclaim (bool &plainMarked, bool &htmlMarked) {
// ================================================================================ // ================================================================================
bool bool
Attachment::markSignificant (const string &parentMultiProp, const streamoff &minAttachSize, ifstream &mbox, vector<Attachment *> &allMarkedPtrs) { Attachment::markSignificant (const string &parentMultiProp, const bool &parentSigned, const streamoff &minAttachSize, ifstream &mbox, vector<Attachment *> &allMarkedPtrs) {
DEF_LOG ("Attachment::markSignificant", "parentMultiProp: " << parentMultiProp << " minAttachSize: " << minAttachSize); DEF_LOG ("Attachment::markSignificant", "parentMultiProp: " << parentMultiProp << " minAttachSize: " << minAttachSize);
isSigned |= parentSigned;
string textProp = getProp (contentTypeToken, textRegEx); string textProp = getProp (contentTypeToken, textRegEx);
bool cantBeExtract ((parentMultiProp == ALTERNATIVE && (textProp == PLAIN || textProp == HTML)) || bool cantBeExtract ((parentMultiProp == ALTERNATIVE && (textProp == PLAIN || textProp == HTML)) ||
(parentMultiProp == RELATED && textProp == HTML)); (parentMultiProp == RELATED && textProp == HTML));
string multiProp = getProp (contentTypeToken, multiRegEx); string multiProp = getProp (contentTypeToken, multiRegEx);
for (Attachment &sub : subAttachements) for (Attachment &sub : subAttachements)
cantBeExtract |= sub.markSignificant (multiProp, minAttachSize, mbox, allMarkedPtrs); cantBeExtract |= sub.markSignificant (multiProp, parentSigned || isSigned, minAttachSize, mbox, allMarkedPtrs);
if (getProp (contentTypeToken, textRegEx) == HTML) { if (getProp (contentTypeToken, textRegEx) == HTML) {
if (KAZ_ATTACH_NAME == getAttachName ())
isKazAttachment = true;
else {
string content = getContent (mbox); string content = getContent (mbox);
vector<string> imgs; vector<string> imgs;
getSection (content, IMG_BEGIN, IMG_END, imgs); getSection (content, IMG_BEGIN, IMG_END, imgs);
@ -424,18 +486,30 @@ Attachment::markSignificant (const string &parentMultiProp, const streamoff &min
if (embeddedData.size ()) if (embeddedData.size ())
toUpdate = true; toUpdate = true;
} }
}
cantBeExtract |= toUpdate; cantBeExtract |= toUpdate;
if (boundary.empty () && getSize () >= minAttachSize && !cantBeExtract) if (boundary.empty () && getSize () >= minAttachSize && !cantBeExtract)
cantBeExtract = toExtract = true; // XXX cantBeExtract ? cantBeExtract = toExtract = true;
if (toExtract || toUpdate || toDisclaim) if (toExtract || toUpdate || toDisclaim || isKazAttachment)
allMarkedPtrs.push_back (this); allMarkedPtrs.push_back (this);
return cantBeExtract; return cantBeExtract;
} }
// ================================================================================
string
Attachment::getMime (ifstream &mbox) const {
DEF_LOG ("Attachment::getMime", "beginPos: " << beginPos << " contentPos: " << contentPos);
string mime;
mime.resize (contentPos-beginPos);
mbox.seekg (beginPos, ios::beg);
mbox.read (&mime[0], contentPos-beginPos);
return mime;
}
// ================================================================================ // ================================================================================
string string
Attachment::getContent (ifstream &mbox) const { Attachment::getContent (ifstream &mbox) const {
DEF_LOG ("Attachment::getContent", "contentPos: " << contentPos); DEF_LOG ("Attachment::getContent", "contentPos: " << contentPos << " endPos: " << endPos);
string content; string content;
content.resize (endPos-contentPos); content.resize (endPos-contentPos);
mbox.seekg (contentPos, ios::beg); mbox.seekg (contentPos, ios::beg);
@ -450,7 +524,7 @@ Attachment::getContent (ifstream &mbox) const {
// ================================================================================ // ================================================================================
void void
Attachment::println (ofstream &outbox, string content) const { Attachment::println (ofstream &outbox, string content) const {
DEF_LOG ("Attachment::println", "content: " << content); DEF_LOG ("Attachment::println", "content: " << content.substr (0, 100) << "...");
if (isBase64Encoding ()) if (isBase64Encoding ())
base64Encode (content); base64Encode (content);
if (isQuotedPrintableEnconding ()) if (isQuotedPrintableEnconding ())
@ -493,7 +567,9 @@ Attachment::replaceEmbedded (string &content) const {
ostream& ostream&
kaz::operator << (ostream& os, const Attachment& attachment) { kaz::operator << (ostream& os, const Attachment& attachment) {
string prop, sep; string prop, sep;
if (attachment.toExtract) { prop = "to extract"; sep = ", "; } if (attachment.isSigned) { prop += sep+"signed"; sep = ", "; }
if (attachment.isKazAttachment) { prop += sep+"kazDisclaim"; sep = ", "; }
if (attachment.toExtract) { prop += sep+"to extract"; sep = ", "; }
if (attachment.toUpdate) { prop += sep+"need update"; sep = ", "; } if (attachment.toUpdate) { prop += sep+"need update"; sep = ", "; }
if (attachment.toDisclaim) { prop += sep+"need diclaim"; sep = ", "; } if (attachment.toDisclaim) { prop += sep+"need diclaim"; sep = ", "; }
if (attachment.embeddedData.size ()) { prop += sep+"embeddedData"; } if (attachment.embeddedData.size ()) { prop += sep+"embeddedData"; }

View File

@ -42,6 +42,10 @@
#include <unistd.h> #include <unistd.h>
#include <vector> #include <vector>
#include <boost/assign.hpp>
#include <boost/algorithm/string.hpp>
#include "version.hpp"
#include "kazDebug.hpp" #include "kazDebug.hpp"
#include "kazMisc.hpp" #include "kazMisc.hpp"
#include "SizeArg.hpp" #include "SizeArg.hpp"
@ -57,13 +61,12 @@ static const string TMPL_FILENAME ("{{FILENAME}}");
static const string CID ("cid:"); static const string CID ("cid:");
// "l=/" => v1 compatibility // "l=/" => v1 compatibility
static const regex archiveURLSignature (".*(([&?]g=)|([&?]l=/)).*"); static const regex archiveURLRegex (".*(([&?]g=)|([&?]l=/)).*");
static const string KAZ_PLAIN_HR ("______________________________________________________________________________"); static const string KAZ_PLAIN_HR ("______________________________________________________________________________");
static const string KAZ_PLAIN_START ("~~ PJ-KAZ !"); // don't end whith space static const string KAZ_PLAIN_START ("~~ PJ-KAZ !"); // don't end whith space
static const string KAZ_PLAIN_STOP (KAZ_PLAIN_START+" ~~"); static const string KAZ_PLAIN_STOP (KAZ_PLAIN_START+" ~~");
static const string KAZ_PLAIN_DONT_TOUCH ("(conservez cette partie intacte dans votre réponse si vous voulez transmettre les documents précédents)"); static const string KAZ_PLAIN_DONT_TOUCH ("(conservez cette partie intacte dans votre réponse si vous voulez transmettre les documents précédents (version "+LAST_VERSION_NUM+"))");
static const string KAZ_PLAIN_WARNING ("Attention : Kaz a dépollué ce message. Les pièces jointes ont été retirées et placées dans un dépôt provisoire. " static const string KAZ_PLAIN_WARNING ("Attention : Kaz a dépollué ce message. Les pièces jointes ont été retirées et placées dans un dépôt provisoire. "
"Elles seront automatiquement supprimées dans 1 mois. " "Elles seront automatiquement supprimées dans 1 mois. "
"Si elles sont importantes et que vous souhaitez les conserver, vous devez utiliser les liens ci-dessous. " "Si elles sont importantes et que vous souhaitez les conserver, vous devez utiliser les liens ci-dessous. "
@ -83,6 +86,7 @@ static const string LI_ONE (LI_BEGIN+" "+CLASS_ONE+">");
static const string LI_ALL (LI_BEGIN+" class=\"all\">"); static const string LI_ALL (LI_BEGIN+" class=\"all\">");
static const string LI_END ("</li>"); static const string LI_END ("</li>");
static const string HREF_ONE ("href=\""); static const string HREF_ONE ("href=\"");
static const string KAZ_HTML_CONTENT ("<!DOCTYPE html><html lang=\"fr\"><head><meta charset=\"utf-8\"><title>KAZ</title>"+KAZ_CSS+"</head><body>");
static const string BODY_END ("</body>"); static const string BODY_END ("</body>");
static const string HTML_END ("</html>"); static const string HTML_END ("</html>");
@ -90,7 +94,7 @@ static const string KAZ_HTML_TAG ("<!--KAZ"); // don't end whith space
static const string KAZ_HTML_START (KAZ_HTML_TAG+" START-->"); static const string KAZ_HTML_START (KAZ_HTML_TAG+" START-->");
static const string KAZ_HTML_STOP (KAZ_HTML_TAG+" STOP-->"); static const string KAZ_HTML_STOP (KAZ_HTML_TAG+" STOP-->");
// Textes précédents encodés en SGML // Textes précédents encodés en SGML
static const string KAZ_HTML_DONT_TOUCH ("(conservez cette partie intacte dans votre r&eacute;ponse si vous voulez transmettre les documents pr&eacute;c&eacute;dents)"); static const string KAZ_HTML_DONT_TOUCH ("(conservez cette partie intacte dans votre r&eacute;ponse si vous voulez transmettre les documents pr&eacute;c&eacute;dents (version "+LAST_VERSION_NUM+"))");
static const string KAZ_HTML_DOWLOAD_ONE ("Vos pi&egrave;ces jointes sont &agrave; t&eacute;l&eacute;charger individuellement ici :"); static const string KAZ_HTML_DOWLOAD_ONE ("Vos pi&egrave;ces jointes sont &agrave; t&eacute;l&eacute;charger individuellement ici :");
static const string KAZ_HTML_DOWLOAD_OTHER ("(Contenu dans des messages pr&eacute;c&eacute;dents)"); static const string KAZ_HTML_DOWLOAD_OTHER ("(Contenu dans des messages pr&eacute;c&eacute;dents)");
static const string KAZ_HTML_DOWLOAD_ALL ("Vous pouvez t&eacute;l&eacute;charger l'ensemble dans une archive l&agrave; :"); static const string KAZ_HTML_DOWLOAD_ALL ("Vous pouvez t&eacute;l&eacute;charger l'ensemble dans une archive l&agrave; :");
@ -98,6 +102,9 @@ static const string KAZ_HTML_ARCHIVE ("archive");
static const string KAZ_EMPTY_TEXT_PLAIN ("Content-Type: text/plain; charset=utf-8\n" static const string KAZ_EMPTY_TEXT_PLAIN ("Content-Type: text/plain; charset=utf-8\n"
"Content-Transfer-Encoding: base64\n"); "Content-Transfer-Encoding: base64\n");
static const string KAZ_ATTACHMENT_TEXT_HTML ("Content-Type: text/html; charset=utf-8\n"
"Content-Disposition: attachment; filename=\"" + Attachment::KAZ_ATTACH_NAME + "\"\n"
"Content-Transfer-Encoding: base64\n");
// ================================================================================ // ================================================================================
@ -105,8 +112,8 @@ vector <string>
Attachment::stringsToUpdate ({KAZ_PLAIN_START, "\""+CID}); Attachment::stringsToUpdate ({KAZ_PLAIN_START, "\""+CID});
// ================================================================================ // ================================================================================
const string MainAttachment::templatePlainAddLink (" * "+TMPL_FILENAME+" <"+TMPL_DOWNLOAD+">\r\n"); const string MainAttachment::templatePlainAddLink (" * "+TMPL_FILENAME+" < "+TMPL_DOWNLOAD+" >\n");
const string MainAttachment::templatePlainAllLink ("\r\n * "+KAZ_PLAIN_DOWLOAD_ALL+" <"+TMPL_DOWNLOAD+">\r\n"); const string MainAttachment::templatePlainAllLink ("\n * "+KAZ_PLAIN_DOWLOAD_ALL+" < "+TMPL_DOWNLOAD+" >\n");
const string MainAttachment::templateHtmlHeader (KAZ_HTML_START+"<p style=\"clear: left; padding: 1pc 0 0 0; font-size:10px; color:#969696;\">"+KAZ_PLAIN_START+"</p><hr>\n" const string MainAttachment::templateHtmlHeader (KAZ_HTML_START+"<p style=\"clear: left; padding: 1pc 0 0 0; font-size:10px; color:#969696;\">"+KAZ_PLAIN_START+"</p><hr>\n"
"<div class=\"kaz\">" "<div class=\"kaz\">"
@ -117,10 +124,67 @@ const string MainAttachment::templateHtmlOtherLink ("</ul>"+KAZ_HTML_DOWLOAD_OTH
const string MainAttachment::templateHtmlAllLink ("</ul><ul>"+LI_ALL+KAZ_HTML_DOWLOAD_ALL+" <a href=\""+TMPL_DOWNLOAD+"\">"+KAZ_HTML_ARCHIVE+"</a>"+LI_END+"\n"); const string MainAttachment::templateHtmlAllLink ("</ul><ul>"+LI_ALL+KAZ_HTML_DOWLOAD_ALL+" <a href=\""+TMPL_DOWNLOAD+"\">"+KAZ_HTML_ARCHIVE+"</a>"+LI_END+"\n");
const string MainAttachment::templateHtmlFooter ("</ul></p>\n" const string MainAttachment::templateHtmlFooter ("</ul></p>\n"
"<p class=\"msg\"><a class=\"kaz\" href=\""+KAZ_WEB_SITE+"\"> "+KAZ_WEB_SITE+" </a></p></div>\n" "<p class=\"msg\"><a class=\"kaz\" href=\""+KAZ_WEB_SITE+"\"> "+KAZ_WEB_SITE+" </a></p></div>\n"
"<hr><p style=\"font-size:10px; color:#969696;\">"+KAZ_PLAIN_STOP+"</p>"+KAZ_HTML_STOP+"\n"); "<hr><p style=\"font-size:10px; color:#969696;\">"+KAZ_PLAIN_STOP+"</p>"+KAZ_HTML_STOP+"\n\n");
const regex MainAttachment::whiteSpaceRegEx ("\\s+"); const regex MainAttachment::whiteSpaceRegEx ("\\s+");
// ================================================================================
const string
kaz::attachModeLabels[] = {
"None", "Footer", "Attachment", "Both"
};
const map<string, AttachMode>
kaz::attachModeMap = boost::assign::map_list_of
("none", NONE)
("footer", FOOTER)
("attachment", ATTACHMENT)
("both", BOTH)
;
ostream &
kaz::operator << (ostream &out, const AttachMode &attachMode) {
//BOOST_ASSERT (treeType >= MIN && treeType <= ALPHA);
return out << attachModeLabels [attachMode];
}
istream &
kaz::operator >> (istream &in, AttachMode &attachMode) {
string token;
in >> token;
auto pos = attachModeMap.find (boost::algorithm::to_lower_copy (token));
if (pos == attachModeMap.end ())
in.setstate (ios_base::failbit);
else
attachMode = pos->second;
return in;
}
// ================================================================================
const string
kaz::headerTypeLabels[] = {
"Same", "Mixed", "MainPlain"
};
const map<string, HeaderType>
kaz::headerTypeMap = boost::assign::map_list_of
("same", SAME)
("mixed", MIXED)
("mainplain", MAIN_PLAIN)
;
ostream &
kaz::operator << (ostream &out, const HeaderType &headerType) {
//BOOST_ASSERT (treeType >= MIN && treeType <= ALPHA);
return out << headerTypeLabels [headerType];
}
istream &
kaz::operator >> (istream &in, HeaderType &headerType) {
string token;
in >> token;
auto pos = headerTypeMap.find (boost::algorithm::to_lower_copy (token));
if (pos == headerTypeMap.end ())
in.setstate (ios_base::failbit);
else
headerType = pos->second;
return in;
}
// ================================================================================ // ================================================================================
void void
MainAttachment::copy (ifstream &mbox, ofstream &outbox, const streamoff &begin, const streamoff &end) { MainAttachment::copy (ifstream &mbox, ofstream &outbox, const streamoff &begin, const streamoff &end) {
@ -141,8 +205,8 @@ MainAttachment::readArchiveUrl () {
archiveDownloadURL.clear (); archiveDownloadURL.clear ();
string line; string line;
getline (cin, line); getline (cin, line);
LOG_BUG (line.rfind ("arch: ", 0) != 0, return, "eMailShrinker: bug ZZ: no archive link. (line: " << line << ")"); LOG_BUG (line.rfind ("arch: ", 0) != 0, return, "eMailShrinker: bug M9: no archive link. (line: " << line << ")");
LOG_BUG (line.rfind ("arch: bad", 0) == 0, return, "eMailShrinker: bug ZZ: bad archive link. (line: " << line << ")"); LOG_BUG (line.rfind ("arch: bad", 0) == 0, return, "eMailShrinker: bug M10: bad archive link. (line: " << line << ")");
if (line.rfind ("arch: none", 0) == 0) if (line.rfind ("arch: none", 0) == 0)
return; return;
archiveDownloadURL = line.substr (6); archiveDownloadURL = line.substr (6);
@ -156,7 +220,7 @@ MainAttachment::readDownloadUrl (string &url) {
string line; string line;
getline (cin, line); getline (cin, line);
LOG ("get URL: " << line); LOG ("get URL: " << line);
LOG_BUG (line.rfind ("url: ", 0) != 0, return, "eMailShrinker: bug ZZ: no download link. (line: " << line << ")"); LOG_BUG (line.rfind ("url: ", 0) != 0, return, "eMailShrinker: bug M11: no download link. (line: " << line << ")");
url = line.substr (5); url = line.substr (5);
} }
@ -179,7 +243,7 @@ MainAttachment::addLink (string &plain, string &html, const string &url, const s
plain += plainNewOneLink; plain += plainNewOneLink;
string htmlNewOneLink (templateHtmlAddLink); string htmlNewOneLink (templateHtmlAddLink);
string codedUrl (url); string codedUrl (url);
// XXX amp ? // pb &amp;
// replaceAll (codedUrl, "&", "&amp;"); // replaceAll (codedUrl, "&", "&amp;");
replaceAll (htmlNewOneLink, TMPL_DOWNLOAD, codedUrl); replaceAll (htmlNewOneLink, TMPL_DOWNLOAD, codedUrl);
replaceAll (htmlNewOneLink, TMPL_FILENAME, name); replaceAll (htmlNewOneLink, TMPL_FILENAME, name);
@ -189,13 +253,13 @@ MainAttachment::addLink (string &plain, string &html, const string &url, const s
// ================================================================================ // ================================================================================
void void
MainAttachment::getDisclaim (string &plain, string &html) const { MainAttachment::getDisclaim (string &plain, string &html) const {
DEF_LOG ("Attachment::getDisclaim", ""); DEF_LOG ("MainAttachment::getDisclaim", "");
plain = html = ""; plain = html = "";
int linkCount (0); int linkCount (0);
string plainNewLinks, htmlNewLinks; string plainNewLinks, htmlNewLinks;
for (Attachment *attachP : allMarkedPtrs) { for (Attachment *attachP : allMarkedPtrs) {
if (!attachP->toExtract) if (attachP->isSigned || !attachP->toExtract)
continue; continue;
addLink (plainNewLinks, htmlNewLinks, attachP->downloadUrl, attachP->getAttachName ()); addLink (plainNewLinks, htmlNewLinks, attachP->downloadUrl, attachP->getAttachName ());
++linkCount; ++linkCount;
@ -204,7 +268,7 @@ MainAttachment::getDisclaim (string &plain, string &html) const {
// previousLinks.erase (attachP->downloadUrl); // previousLinks.erase (attachP->downloadUrl);
} }
for (Attachment *attachP : allMarkedPtrs) { for (Attachment *attachP : allMarkedPtrs) {
if (!attachP->embeddedData.size ()) if (attachP->isSigned || !attachP->embeddedData.size ())
continue; continue;
for (EmbeddedData &embedded : attachP->embeddedData) { for (EmbeddedData &embedded : attachP->embeddedData) {
addLink (plainNewLinks, htmlNewLinks, embedded.downloadUrl, embedded.name); addLink (plainNewLinks, htmlNewLinks, embedded.downloadUrl, embedded.name);
@ -221,10 +285,10 @@ MainAttachment::getDisclaim (string &plain, string &html) const {
return; return;
} }
plain = "\r\n"+KAZ_PLAIN_START+"\r\n"+KAZ_PLAIN_HR+"\r\n"+KAZ_PLAIN_DONT_TOUCH+"\r\n\r\n"+KAZ_PLAIN_WARNING+"\r\n\r\n"+KAZ_PLAIN_DOWLOAD_ONE+"\r\n"+plainNewLinks; plain = "\n"+KAZ_PLAIN_START+"\n\n"+KAZ_PLAIN_HR+"\n"+KAZ_PLAIN_DONT_TOUCH+"\n\n"+KAZ_PLAIN_DOWLOAD_ONE+"\n"+plainNewLinks;
html = templateHtmlHeader+htmlNewLinks; html = templateHtmlHeader+htmlNewLinks;
if (previousLinks.size ()) { if (previousLinks.size ()) {
plain += "\r\n"+KAZ_PLAIN_DOWLOAD_OTHER+"\r\n"+plainOldLinks; plain += "\n"+KAZ_PLAIN_DOWLOAD_OTHER+"\n"+plainOldLinks;
html += templateHtmlOtherLink+htmlOldLinks; html += templateHtmlOtherLink+htmlOldLinks;
} }
if (linkCount > 1 && archiveDownloadURL.length ()) { if (linkCount > 1 && archiveDownloadURL.length ()) {
@ -235,8 +299,8 @@ MainAttachment::getDisclaim (string &plain, string &html) const {
replaceAll (allHtmlLinks, TMPL_DOWNLOAD, archiveDownloadURL); replaceAll (allHtmlLinks, TMPL_DOWNLOAD, archiveDownloadURL);
html += allHtmlLinks; html += allHtmlLinks;
} }
html += templateHtmlFooter+"\r\n"; html += templateHtmlFooter+"\n";
plain += "\r\n\r\n"+KAZ_WEB_SITE+"\r\n"+KAZ_PLAIN_HR+"\r\n"+KAZ_PLAIN_STOP+"\r\n"; plain += "\n\n"+KAZ_WEB_SITE+"\n\n"+KAZ_PLAIN_WARNING+"\n"+KAZ_PLAIN_HR+"\n"+KAZ_PLAIN_STOP+"\n\n\n";
// & => &amp; done // & => &amp; done
LOG ("plain: " << plain); LOG ("plain: " << plain);
LOG ("html: " << html); LOG ("html: " << html);
@ -245,21 +309,21 @@ MainAttachment::getDisclaim (string &plain, string &html) const {
// ================================================================================ // ================================================================================
void void
MainAttachment::addPrevious (const string &href, const string &name, const bool &trust) { MainAttachment::addPrevious (const string &href, const string &name, const bool &trust) {
DEF_LOG ("Attachment::addPrevious", "href: " << href << " name: " << name); DEF_LOG ("MainAttachment::addPrevious", "href: " << href << " name: " << name);
const string oldVal = previousLinks [href]; const string oldVal = previousLinks [href];
if (name.empty ()) if (name.empty ())
return; return;
if (oldVal.length () && name.length () && !trust) if (oldVal.length () && name.length () && !trust)
return; return;
previousLinks.erase (href); previousLinks.erase (href);
previousLinks [href] = regex_replace (name, regex (R"([\t\r\n\"]+|(\\\")|(>\s*))"), ""); previousLinks [href] = regex_replace (name, regex (R"([\t\n\"]+|(\\\")|(>\s*))"), "");
LOG ("inserted: " << href << ": " << previousLinks[href]); LOG ("inserted: " << href << ": " << previousLinks[href]);
} }
void void
MainAttachment::extractLinks (const string &extractedPlainKAZ) { MainAttachment::extractLinks (const string &extractedPlainKAZ) {
// plain text => "* name <url>" // plain text => "* name <url>"
DEF_LOG ("Attachment::extractedPlainKAZ", "extractedPlainKAZ: " << extractedPlainKAZ); DEF_LOG ("MainAttachment::extractedPlainKAZ", "extractedPlainKAZ: " << extractedPlainKAZ);
for (string::size_type startPos (0); for (string::size_type startPos (0);
(startPos = extractedPlainKAZ.find ("http", startPos)) != string::npos; (startPos = extractedPlainKAZ.find ("http", startPos)) != string::npos;
) { ) {
@ -284,7 +348,7 @@ MainAttachment::extractLinks (const string &extractedPlainKAZ) {
string name; string name;
if (startName != string::npos) { if (startName != string::npos) {
name = string (extractedPlainKAZ, startName+3, stopName - startName - 3); name = string (extractedPlainKAZ, startName+2, stopName -startName-2);
// skip [> \r\n\t] // skip [> \r\n\t]
string::size_type nextPos = name.find_first_not_of ("[>< \t\r\n\"]"); string::size_type nextPos = name.find_first_not_of ("[>< \t\r\n\"]");
if (nextPos != string::npos) if (nextPos != string::npos)
@ -304,7 +368,7 @@ MainAttachment::extractLinks (const string &extractedPlainKAZ) {
void void
MainAttachment::extractLinks (const vector<string> &liOne) { MainAttachment::extractLinks (const vector<string> &liOne) {
// html text => "<li ...><a href="url">name</a>" // html text => "<li ...><a href="url">name</a>"
DEF_LOG ("Attachment::extractedPlainKAZ", "liOne.size: " << liOne.size ()); DEF_LOG ("MainAttachment::extractedPlainKAZ", "liOne.size: " << liOne.size ());
for (const string &one : liOne) { for (const string &one : liOne) {
if (caseInsensitiveFind (one, CLASS_ONE) == string::npos) if (caseInsensitiveFind (one, CLASS_ONE) == string::npos)
continue; continue;
@ -336,29 +400,38 @@ MainAttachment::extractLinks (const vector<string> &liOne) {
} }
void void
MainAttachment::extractPreviousKAZ (ifstream &mbox) { MainAttachment::extractPreviousKAZ (string &extractedPlainKAZ, string &extractedHtmlKAZ, ifstream &mbox, const Attachment &attach) {
DEF_LOG ("MainAttachment::extractPreviousKAZ", ""); DEF_LOG ("MainAttachment::extractPreviousKAZ", "attach:" << attach);
string extractedPlainKAZ, extractedHtmlKAZ; if (!(attach.toUpdate || attach.isKazAttachment)) // isKazAttachment => toUpdate
for (const Attachment *attachP : allMarkedPtrs) { return;
if (!attachP->toUpdate || isBase64Encoding ()) string textProp = attach.getProp (contentTypeToken, textRegEx);
continue;
string textProp = attachP->getProp (contentTypeToken, textRegEx);
if (textProp.empty ()) if (textProp.empty ())
continue; return;
string content (attachP->getContent (mbox)); string content (attach.getContent (mbox));
replaceAll (content, "&amp;", "&");
if (textProp == PLAIN) { if (textProp == PLAIN) {
LOG (PLAIN); LOG (PLAIN);
extractedPlainKAZ += attachP->getSection (content, KAZ_PLAIN_START, KAZ_PLAIN_STOP); extractedPlainKAZ += attach.getSection (content, KAZ_PLAIN_START, KAZ_PLAIN_STOP);
} }
if (textProp == HTML) { if (textProp == HTML) {
LOG (HTML); LOG (HTML);
string section = attachP->getSection (content, KAZ_HTML_START, KAZ_HTML_STOP); string section = attach.getSection (content, KAZ_HTML_START, KAZ_HTML_STOP);
section += attachP->getSection (content, KAZ_PLAIN_START, KAZ_PLAIN_STOP); section += attach.getSection (content, KAZ_PLAIN_START, KAZ_PLAIN_STOP);
// update href from HTML attachments // update href from HTML attachments
replaceAll (section, "&amp;", "&");
extractedHtmlKAZ += section; extractedHtmlKAZ += section;
} }
} }
void
MainAttachment::extractPreviousKAZ (ifstream &mbox) {
DEF_LOG ("MainAttachment::extractPreviousKAZ", "");
string extractedPlainKAZ, extractedHtmlKAZ;
if (boundary.empty ())
extractPreviousKAZ (extractedPlainKAZ, extractedHtmlKAZ, mbox, *this);
else
for (const Attachment *attachP : allMarkedPtrs)
extractPreviousKAZ (extractedPlainKAZ, extractedHtmlKAZ, mbox, *attachP);
LOG ("extractedPlainKAZ: "<< extractedPlainKAZ); LOG ("extractedPlainKAZ: "<< extractedPlainKAZ);
extractLinks (extractedPlainKAZ); extractLinks (extractedPlainKAZ);
@ -375,20 +448,81 @@ MainAttachment::extractPreviousKAZ (ifstream &mbox) {
void void
MainAttachment::removePreviousArchive () { MainAttachment::removePreviousArchive () {
DEF_LOG ("MainAttachment::removePreviousArchive", "");
vector<string> toRemove; vector<string> toRemove;
for (map <string, string>::const_iterator it = previousLinks.begin (); it != previousLinks.end (); ++it) { for (map <string, string>::const_iterator it = previousLinks.begin (); it != previousLinks.end (); ++it) {
const string key (it->first); const string key (it->first);
if (regex_match (key, archiveURLSignature)) if (regex_match (key, archiveURLRegex))
toRemove.push_back (key); toRemove.push_back (key);
} }
for (string old : toRemove) for (string old : toRemove)
previousLinks.erase (old); previousLinks.erase (old);
} }
// ================================================================================
void MainAttachment::rewriteHeaders (ifstream &mbox, ofstream &outbox, const HeaderType &headerType) {
DEF_LOG ("MainAttachment::rewriteHeaders", "headerType: " << headerType);
switch (headerType) {
case SAME:
copy (mbox, outbox, 0, contentPos);
return;
case MAIN_PLAIN:
break;
case MIXED:
if (boost::iequals (getContentType (), "multipart/mixed")) {
copy (mbox, outbox, 0, contentPos);
return;
}
break;
}
string mime (getMime (mbox));
mime.insert (0, "\n");
string::size_type startPos = (0);
// remove previous content-type at begin of ligne (after a "\n")
for (string token : {string ("content-transfer-encoding"),
Attachment::contentTypeToken}) {
LOG ("token:" << token);
startPos = caseInsensitiveFind (mime, string ("\n")+token);
if (startPos == string::npos)
continue;
for (string::size_type stopPos (startPos+1);
(stopPos = mime.find ("\n", stopPos)) != string::npos;
) {
if (string (" \t").find (mime [stopPos+1]) == string::npos) {
// not wrap after
movedContentType += mime.substr (startPos+1, stopPos-startPos);
mime.erase (startPos, stopPos-startPos);
break;
}
// merge wrap lines
// find next endl
++stopPos;
}
}
string contentType (KAZ_EMPTY_TEXT_PLAIN);
switch (headerType) {
case SAME: /* no way */ break;
case MAIN_PLAIN: contentType = KAZ_EMPTY_TEXT_PLAIN; break;
case MIXED:
addedBoundary = "__KAZ__"+boundaryGen (40);
contentType = "Content-Type: multipart/mixed; boundary=\""+addedBoundary+"\"\n";
addedBoundary = "--"+addedBoundary+"--";
addedBoundaryMiddleSize = addedBoundary.length () - 2;
}
// skip '\n'
++startPos;
if (startPos >= mime.length ())
startPos = mime.length () - 1;
mime.insert (startPos, contentType);
outbox << mime.substr (1) << endl << flush;
}
// ================================================================================ // ================================================================================
MainAttachment::MainAttachment (ifstream &mbox) MainAttachment::MainAttachment (ifstream &mbox)
: Attachment (mbox, initTmpLevel (), 0, initTmpPos ()), : Attachment (mbox, initTmpLevel (), 0, initTmpPos ()),
forceMainText (false) { emptyEMail (false),
previousKazAttachment (false) {
DEF_LOG ("MainAttachment::MainAttachment", ""); DEF_LOG ("MainAttachment::MainAttachment", "");
string line; string line;
for (; getline (mbox, line); ) for (; getline (mbox, line); )
@ -402,8 +536,8 @@ MainAttachment::markSignificant (const streamoff &minAttachSize, ifstream &mbox)
DEF_LOG ("MainAttachment::markSignificant", "minAttachSize: " << minAttachSize); DEF_LOG ("MainAttachment::markSignificant", "minAttachSize: " << minAttachSize);
bool plainMarked (false), htmlMarked (false); bool plainMarked (false), htmlMarked (false);
markDisclaim (plainMarked, htmlMarked); markDisclaim (plainMarked, htmlMarked);
forceMainText = ! (plainMarked || htmlMarked); emptyEMail = ! (plainMarked || htmlMarked);
Attachment::markSignificant ("", minAttachSize, mbox, allMarkedPtrs); Attachment::markSignificant ("", isSigned, minAttachSize, mbox, allMarkedPtrs);
} }
// ================================================================================ // ================================================================================
@ -446,7 +580,7 @@ MainAttachment::extract (ifstream &mbox, const SizeArg &minSize) const {
int attachCount (0); int attachCount (0);
string dirName, mediaName; string dirName, mediaName;
for (Attachment *attachP : allMarkedPtrs) { for (Attachment *attachP : allMarkedPtrs) {
if (!attachP->toExtract) if (attachP->isSigned || attachP->isKazAttachment || !attachP->toExtract)
continue; continue;
newPjEntry (attachCount, attachP->getContentType (), attachP->getAttachName (), dirName, mediaName); newPjEntry (attachCount, attachP->getContentType (), attachP->getAttachName (), dirName, mediaName);
++attachCount; ++attachCount;
@ -497,7 +631,7 @@ MainAttachment::extract (ifstream &mbox, const SizeArg &minSize) const {
cout << dirName << endl; cout << dirName << endl;
} }
for (Attachment *attachP : allMarkedPtrs) { for (Attachment *attachP : allMarkedPtrs) {
if (!attachP->embeddedData.size ()) if (attachP->isSigned || !attachP->embeddedData.size ())
continue; continue;
string content = attachP->getContent (mbox); string content = attachP->getContent (mbox);
vector<string> imgs; vector<string> imgs;
@ -520,16 +654,25 @@ MainAttachment::extract (ifstream &mbox, const SizeArg &minSize) const {
} }
// ================================================================================ // ================================================================================
void /*!
MainAttachment::substitute (ifstream &mbox, ofstream &outbox, const SizeArg &minSize) { Régle à appliquer dans le cas Kaz ajoute son cartouche et que le corps principale n'est pas multipart :
DEF_LOG ("MainAttachment::substitute", "minSize: " << minSize);
// preparation <table>
<tr><th>src</th><th>FOOTER</th><th>BOTH</th><th>ATTCH</th></tr>
<tr><th>text/plain</th><td>OK</td><td>mute multi</td><td>mute multi</td></tr>
<tr><th>empty mail</th><td>mute plain</td><td>mute multi</td><td>mute html</td></tr>
</table>
*/
void
MainAttachment::substitute (ifstream &mbox, ofstream &outbox, const SizeArg &minSize, AttachMode attachMode) {
DEF_LOG ("MainAttachment::substitute", "minSize: " << minSize << " AttachMode: " << attachMode);
// setup
extractPreviousKAZ (mbox); extractPreviousKAZ (mbox);
removePreviousArchive (); removePreviousArchive ();
map<const string, const string> translateHtml; map<const string, const string> translateHtml;
for (Attachment *attachP : allMarkedPtrs) for (Attachment *attachP : allMarkedPtrs)
if (attachP->toExtract) { if (!attachP->isSigned && attachP->toExtract && !attachP->isKazAttachment) {
readDownloadUrl (attachP->downloadUrl); readDownloadUrl (attachP->downloadUrl);
if (attachP->downloadUrl.empty ()) { if (attachP->downloadUrl.empty ()) {
LOG ("no change"); LOG ("no change");
@ -543,7 +686,7 @@ MainAttachment::substitute (ifstream &mbox, ofstream &outbox, const SizeArg &min
} }
} }
for (Attachment *attachP : allMarkedPtrs) { for (Attachment *attachP : allMarkedPtrs) {
if (!attachP->embeddedData.size ()) if (attachP->isSigned || !attachP->embeddedData.size ())
continue; continue;
for (EmbeddedData &embedded : attachP->embeddedData) for (EmbeddedData &embedded : attachP->embeddedData)
readDownloadUrl (embedded.downloadUrl); readDownloadUrl (embedded.downloadUrl);
@ -553,29 +696,82 @@ MainAttachment::substitute (ifstream &mbox, ofstream &outbox, const SizeArg &min
removePreviousArchive (); removePreviousArchive ();
string plainDisclaim, htmlDisclaim; string plainDisclaim, htmlDisclaim;
getDisclaim (plainDisclaim, htmlDisclaim); getDisclaim (plainDisclaim, htmlDisclaim);
LOG_BUG (plainDisclaim.empty () ^ htmlDisclaim.empty (), /* */,
"eMailShrinker: bug M8: not empty disclaim. (plainDisclaim: " << plainDisclaim.size () << " htmlDisclaim: " << htmlDisclaim.size () << ")");
if (plainDisclaim.empty ()) {
// no change
cerr << "eMailShrinker: original email inchanged" << endl;
copy (mbox, outbox, 0, endPos);
return;
}
if (boost::iequals (getContentType (), "multipart/report")) {
// no change
cerr << "eMailShrinker: repport email inchange" << endl;
copy (mbox, outbox, 0, endPos);
return;
}
HeaderType headerType (SAME);
// copy email // copy email
streamoff curPos = 0; switch (attachMode) {
if (forceMainText) { case NONE: LOG_BUG (true, /* */, "eMailShrinker: bug M12: nothing to do"); break;
cerr << endl << endl << " #################### coucou " << forceMainText << " " << contentPos << " " << *this << endl; case FOOTER: headerType = (emptyEMail && !boundary.size ()) ? MAIN_PLAIN : SAME; break;
// check no main text case BOTH: headerType = MIXED; break;
LOG ("Force main text"); case ATTACHMENT: headerType = MIXED; break;
LOG_BUG (boundary.empty () || ! subAttachements.size (), /**/, "eMailShrinker: can't force add footer M9: : " << *this); }
copy (mbox, outbox, curPos, contentPos); rewriteHeaders (mbox, outbox, headerType);
curPos = contentPos; streamoff curPos = contentPos;
cerr << " #################### coucou " << curPos << endl << endl;
if (MAIN_PLAIN == headerType) {
LOG ("Replace old content with plain");
string content (plainDisclaim); string content (plainDisclaim);
base64Encode (content); base64Encode (content);
outbox << boundary.substr (0, boundary.length () -2) << endl outbox << content << endl;
outbox.flush ();
return;
}
// // XXX => MIXED
// if (ATTACH_HTML == headerType) {
// LOG ("Replace old content with html");
// string content (plainDisclaim);
// base64Encode (content);
// outbox << content << endl;
// outbox.flush ();
// return;
// }
if (emptyEMail && (attachMode & FOOTER)) {
// case : multi
LOG ("Force main text");
cerr << "eMailShrinker: force main text" << endl;
string content (plainDisclaim);
base64Encode (content);
outbox << boundary.substr (0, boundaryMiddleSize) << endl
<< KAZ_EMPTY_TEXT_PLAIN << endl << KAZ_EMPTY_TEXT_PLAIN << endl
<< content << endl; << content << endl;
outbox.flush (); outbox.flush ();
} }
if (movedContentType.size ()) {
LOG ("New boundary");
outbox << addedBoundary.substr (0, addedBoundaryMiddleSize) << endl
<< movedContentType << endl;
}
for (Attachment *attachP : allMarkedPtrs) { for (Attachment *attachP : allMarkedPtrs) {
copy (mbox, outbox, curPos, attachP->beginInParent); copy (mbox, outbox, curPos, attachP->beginInParent);
outbox << endl; // force end MIME section
LOG_BUG (attachP->toUpdate && attachP->toExtract, /* */, "eMailShrinker: bug M5: update and extract. pos: " << attachP->beginPos); LOG_BUG (attachP->toUpdate && attachP->toExtract, /* */, "eMailShrinker: bug M5: update and extract. pos: " << attachP->beginPos);
if (attachP->toExtract) { if (attachP->isSigned) {
LOG ("skip Extracted"); LOG ("don't change signed content");
copy (mbox, outbox, attachP->beginInParent, attachP->endPos);
} else if (attachP->toExtract || attachP->isKazAttachment) {
LOG ("skip Extracted or previous attachments");
} else if (attachP->toUpdate) { } else if (attachP->toUpdate) {
string textProp = attachP->getProp (contentTypeToken, textRegEx); string textProp = attachP->getProp (contentTypeToken, textRegEx);
@ -586,6 +782,7 @@ MainAttachment::substitute (ifstream &mbox, ofstream &outbox, const SizeArg &min
LOG_BUG (isPlain && isHtml, /* */, "eMailShrinker: bug M6: plain and html: " << attachP->getContentType ()); LOG_BUG (isPlain && isHtml, /* */, "eMailShrinker: bug M6: plain and html: " << attachP->getContentType ());
LOG_BUG (! (isPlain || isHtml), /* */, "eMailShrinker: bug M7: not plain or html: " << attachP->getContentType ()); LOG_BUG (! (isPlain || isHtml), /* */, "eMailShrinker: bug M7: not plain or html: " << attachP->getContentType ());
LOG ("toUpdate: isPlain: " << isPlain << " isHtml: " << isHtml << " isDisclaimer: " << isDisclaimer); LOG ("toUpdate: isPlain: " << isPlain << " isHtml: " << isHtml << " isDisclaimer: " << isDisclaimer);
if (attachP != this)
copy (mbox, outbox, attachP->beginInParent, attachP->contentPos); copy (mbox, outbox, attachP->beginInParent, attachP->contentPos);
string content = attachP->getContent (mbox); string content = attachP->getContent (mbox);
@ -616,14 +813,13 @@ MainAttachment::substitute (ifstream &mbox, ofstream &outbox, const SizeArg &min
} }
removeSection (content, KAZ_HTML_START, KAZ_HTML_STOP); removeSection (content, KAZ_HTML_START, KAZ_HTML_STOP);
removeSection (content, KAZ_PLAIN_START, KAZ_PLAIN_STOP); removeSection (content, KAZ_PLAIN_START, KAZ_PLAIN_STOP);
// XXX case insensitive ?? if (caseInsensitiveFind (content, CID) != string::npos)
if (content.find (CID) != string::npos)
replaceAll (content, translateHtml); replaceAll (content, translateHtml);
attachP->replaceEmbedded (content); attachP->replaceEmbedded (content);
} }
if (isPlain) if (isPlain)
removeSection (content, KAZ_PLAIN_START, KAZ_PLAIN_STOP); removeSection (content, KAZ_PLAIN_START, KAZ_PLAIN_STOP);
if (isDisclaimer) { if (isDisclaimer && (attachMode & FOOTER)) {
if (isHtml) { if (isHtml) {
for (string endTag : {BODY_END, HTML_END}) { for (string endTag : {BODY_END, HTML_END}) {
LOG ("try tag: " << endTag); LOG ("try tag: " << endTag);
@ -641,12 +837,37 @@ MainAttachment::substitute (ifstream &mbox, ofstream &outbox, const SizeArg &min
} }
attachP->println (outbox, content); attachP->println (outbox, content);
} else { } else {
LOG_BUG (true, continue, "eMailShrinker: bug M8: can't change" << *attachP); LOG_BUG (true, continue, "eMailShrinker: upload has failed, so can't change" << *attachP);
} }
outbox.flush (); outbox.flush ();
curPos = attachP->endPos; curPos = attachP->endPos;
} }
if (subAttachements.size ()) {
streamoff lastPos = subAttachements.back ().endPos;
copy (mbox, outbox, curPos, lastPos);
curPos = lastPos;
}
if (attachMode & ATTACHMENT) {
LOG ("Add kaz attachment");
if (movedContentType.size ()) {
copy (mbox, outbox, curPos, endPos); copy (mbox, outbox, curPos, endPos);
outbox << addedBoundary.substr (0, addedBoundaryMiddleSize) << endl
<< KAZ_ATTACHMENT_TEXT_HTML << endl;
} else
outbox << boundary.substr (0, boundaryMiddleSize) << endl
<< KAZ_ATTACHMENT_TEXT_HTML << endl;
cerr << "eMailShrinker: force attachment" << endl;
string content (KAZ_HTML_CONTENT+htmlDisclaim+BODY_END+HTML_END);
base64Encode (content);
outbox << content << endl;
outbox.flush ();
}
if (!movedContentType.size ())
copy (mbox, outbox, curPos, endPos);
else
outbox << addedBoundary << endl;
outbox.close (); outbox.close ();
} }

View File

@ -32,7 +32,10 @@
// knowledge of the CeCILL-B license and that you accept its terms. // // knowledge of the CeCILL-B license and that you accept its terms. //
//////////////////////////////////////////////////////////////////////////// ////////////////////////////////////////////////////////////////////////////
#define LAST_VERSION "2.1 2022-10-30 eMailShrinker" #include "version.hpp"
const std::string kaz::LAST_VERSION_NUM ("2.21");
const std::string kaz::LAST_VERSION_DATE ("2024-06-04");
const std::string kaz::LAST_VERSION (LAST_VERSION_NUM+" "+LAST_VERSION_DATE+" eMailShrinker");
#include <iostream> #include <iostream>
#include <fstream> #include <fstream>
@ -66,8 +69,8 @@ usage (const string &msg = "", const bool &hidden = false) {
cout << endl cout << endl
<< "Usage: " << endl << "Usage: " << endl
<< " A) " << prog << " -u mbox > url-list" << endl << " A) " << prog << " -u mbox > url-list" << endl
<< " B) " << prog << " [-s size] [-d dirName}] mbox > file-list" << endl << " B) " << prog << " [-s size] [-d dirName] mbox > file-list" << endl
<< " C) " << prog << " [-s size] mbox altered-mbox < url-list" << endl << " C) " << prog << " [-s size] [-m {Footer|Attachment|Both}] mbox altered-mbox < url-list" << endl
<< endl << " filter attachments" << endl << endl << endl << " filter attachments" << endl << endl
<< " A: list previous embded url need to be refresh (no added option)" << endl << " A: list previous embded url need to be refresh (no added option)" << endl
<< " => downloadURL list" << endl << " => downloadURL list" << endl
@ -103,20 +106,21 @@ static const char *const inputFileC = inputFile.c_str ();
int int
main (int argc, char** argv) { main (int argc, char** argv) {
// XXX debug before parse options // uncomment next line in case of debug parse options
// Log::debug = true; // Log::debug = true;
DEF_LOG ("main:", ""); DEF_LOG ("main:", "");
prog = argv [0]; prog = argv [0];
bool bool
debugFlag (false),
helpFlag (false), helpFlag (false),
versionFlag (false), versionFlag (false),
updateListFlag (false), updateListFlag (false),
useTheForceLuke (false), useTheForceLuke (false),
listFlag (false); listFlag (false),
debugFlag (false);
string inputName, outputName; string inputName, outputName;
bfs::path extractDir (bfs::temp_directory_path ()); bfs::path extractDir (bfs::temp_directory_path ());
SizeArg minAttachSize ("48 Ki"); SizeArg minAttachSize ("48 Ki");
AttachMode attachMode (FOOTER);
try { try {
mainDescription.add_options () mainDescription.add_options ()
@ -125,6 +129,7 @@ main (int argc, char** argv) {
("size,s", value<SizeArg> (&minAttachSize)->default_value (minAttachSize), "minimum size for extration") ("size,s", value<SizeArg> (&minAttachSize)->default_value (minAttachSize), "minimum size for extration")
("updateList,u", bool_switch (&updateListFlag), "list URL need refresh") ("updateList,u", bool_switch (&updateListFlag), "list URL need refresh")
("extractDir,d", value<bfs::path> (&extractDir)->default_value (extractDir), "set tmp directory name for extraction") ("extractDir,d", value<bfs::path> (&extractDir)->default_value (extractDir), "set tmp directory name for extraction")
("mode,m", boost::program_options::value<AttachMode> (&attachMode)->default_value (attachMode), "set attachment mode")
; ;
hide.add_options () hide.add_options ()
@ -197,9 +202,11 @@ main (int argc, char** argv) {
attachment.markSignificant (minAttachSize, mbox); attachment.markSignificant (minAttachSize, mbox);
mbox.close (); mbox.close ();
if (listFlag) if (listFlag) {
// debug // debug
cerr << attachment; cerr << attachment;
return 0;
}
if (updateListFlag) { if (updateListFlag) {
// case update // case update
@ -219,9 +226,13 @@ main (int argc, char** argv) {
} }
// case substitute // case substitute
if (attachMode == NONE) {
cerr << endl << prog << ": attachMode can't be NONE (forced FOOTER mode)" << endl;
attachMode = FOOTER;
}
mbox.open (inputName); mbox.open (inputName);
ofstream outbox (outputName); ofstream outbox (outputName);
attachment.substitute (mbox, outbox, minAttachSize); attachment.substitute (mbox, outbox, minAttachSize, attachMode);
showTime ("Substitution"); showTime ("Substitution");
return 0; return 0;
} }

View File

@ -1,294 +0,0 @@
////////////////////////////////////////////////////////////////////////////
// Copyright KAZ 2021 //
// //
// contact (at) kaz.bzh //
// //
// This software is a filter to shrink email by attachment extraction. //
// //
// This software is governed by the CeCILL-B license under French law and //
// abiding by the rules of distribution of free software. You can use, //
// modify and/or redistribute the software under the terms of the //
// CeCILL-B license as circulated by CEA, CNRS and INRIA at the following //
// URL "http://www.cecill.info". //
// //
// As a counterpart to the access to the source code and rights to copy, //
// modify and redistribute granted by the license, users are provided //
// only with a limited warranty and the software's author, the holder of //
// the economic rights, and the successive licensors have only limited //
// liability. //
// //
// In this respect, the user's attention is drawn to the risks associated //
// with loading, using, modifying and/or developing or reproducing the //
// software by the user in light of its specific status of free software, //
// that may mean that it is complicated to manipulate, and that also //
// therefore means that it is reserved for developers and experienced //
// professionals having in-depth computer knowledge. Users are therefore //
// encouraged to load and test the software's suitability as regards //
// their requirements in conditions enabling the security of their //
// systems and/or data to be ensured and, more generally, to use and //
// operate it in the same conditions as regards security. //
// //
// The fact that you are presently reading this means that you have had //
// knowledge of the CeCILL-B license and that you accept its terms. //
////////////////////////////////////////////////////////////////////////////
#define LAST_VERSION "1.1 2022-10-30 jirafeauAPI"
#include <iostream>
#include <string>
#include <curl/curl.h>
#include <chrono>
#include <boost/program_options.hpp>
#include <boost/filesystem.hpp>
#include "kazDebug.hpp"
#include "kazMisc.hpp"
#include "SizeArg.hpp"
using namespace std;
using namespace boost;
using namespace boost::program_options;
using namespace kaz;
namespace bfs = boost::filesystem;
// ================================================================================
static options_description mainDescription ("Main options", getCols ());
static options_description hide ("Hidded options", getCols ());
static char *prog = NULL;
// ================================================================================
void
usage (const string &msg = "", const bool &hidden = false) {
if (!msg.empty ()) {
cout << msg << endl;
exit (1);
}
cout << endl
<< "Usage: " << endl
<< " A) " << prog << " [-s size] [-t period] [-c content-type] [-n attachName] [-f server] send file [password] > url,delCode" << endl
<< " B) " << prog << " [-t period] [-f server] update ref > dealine" << endl
<< endl << " store ficle" << endl << endl
<< " A: send file (options : s, t)" << endl
<< " B: update deadline (options : t) " << endl
<< endl << mainDescription
<< endl;
if (hidden)
cout << hide << endl;
exit (0);
}
void
version () {
cout << LAST_VERSION << " KAZ team production (https://kaz.bzh/)" << endl;
exit (0);
}
static auto startPrg = std::chrono::high_resolution_clock::now ();
void
showTime (string msg) {
using namespace std::chrono;
static auto stopPrg = high_resolution_clock::now ();
cerr << msg << " done in " << ns2string (duration_cast<duration<double> > (stopPrg-startPrg).count ()) << endl;
}
// ================================================================================
static size_t
WriteCallback (void *contents, size_t size, size_t nmemb, void *userp) {
((std::string*) userp)->append ((char*) contents, size * nmemb);
return size * nmemb;
}
// ================================================================================
static const string inputFile = "input-file";
static const char *const inputFileC = inputFile.c_str ();
int
main (int argc, char** argv) {
// XXX debug before parse options
// Log::debug = true;
DEF_LOG ("main:", "");
prog = argv [0];
bool
debugFlag (false),
helpFlag (false),
versionFlag (false),
useTheForceLuke (false);
enum JirCmd { SEND, UPDATE } jirCmd;
string
inputFileName,
password,
contentType,
attachName,
urlBase ("http://file.kaz.bzh"),
uploadPage ("/a.php"),
updatePage ("/a.php"),
minimumAvailability ("month"),
proxy;
SizeArg maxUploadSize ("100 Mi");
try {
mainDescription.add_options ()
("help,h", bool_switch (&helpFlag), "produce this help message")
("version,v", bool_switch (&versionFlag), "display version information")
("contentType,c", value<string> (&contentType)->default_value (contentType), "content-type of the sended file")
("attachName,n", value<string> (&attachName)->default_value (attachName), "force attachment name")
("minimumAvailability,t", value<string> (&minimumAvailability)->default_value (minimumAvailability), "minimum period of available download")
("maxUploadSize,s", value<SizeArg> (&maxUploadSize)->default_value (maxUploadSize), "maximum upload size")
("file server registery,f", value<string> (&urlBase)->default_value (urlBase), "server where file are temporary stored")
;
hide.add_options ()
("useTheForceLuke", bool_switch (&useTheForceLuke), "display hidded options")
("debug,g", bool_switch (&debugFlag), "debug mode")
("proxy,p", value<string> (&proxy)->default_value (proxy), "set proxy (proxy-host.org:8080)")
("uploadPage,u", value<string> (&uploadPage)->default_value (uploadPage), "upload page")
("updatePage,v", value<string> (&updatePage)->default_value (updatePage), "update page")
;
options_description cmd ("All options");
cmd.add (mainDescription).add (hide).add_options ()
(inputFileC, value<vector<string> > (), "input")
;
positional_options_description p;
p.add (inputFileC, -1);
variables_map vm;
basic_parsed_options<char> parsed = command_line_parser (argc, argv).options (cmd).positional (p).run ();
store (parsed, vm);
notify (vm);
if (debugFlag) {
#ifdef DISABLE_LOG
cerr << "No debug option available (was compiled with -DDISABLE_LOG)" << endl;
#endif
}
Log::debug = debugFlag;
if (useTheForceLuke)
usage ("", true);
if (versionFlag)
version ();
if (helpFlag)
usage ();
if (vm.count (inputFileC)) {
vector<string> var = vm[inputFileC].as<vector<string> > ();
int nbArgs = vm[inputFileC].as<vector<string> > ().size ();
if (!nbArgs)
usage ("No command");
if (var [0].compare ("send") == 0)
jirCmd = SEND;
else if (var [0].compare ("update") == 0)
jirCmd = UPDATE;
else
usage ("Unknown command ("+var [0]+")");
if (nbArgs < 2)
usage ("no input file");
inputFileName = var [1];
if (nbArgs == 3)
password = var [2];
if (nbArgs > 3)
usage ("Too much arguments");
}
} catch (std::exception &e) {
cerr << "error: " << e.what() << endl;
usage ();
return 1;
} catch (...) {
cerr << "Exception of unknown type!" << endl;
return 1;
}
if (inputFileName.empty ())
usage ("no input");
CURL *easyhandle = curl_easy_init ();
if (! easyhandle) {
cerr << "no curl" << endl;
return 1;
}
string readBuffer;
if (proxy.length ())
curl_easy_setopt(easyhandle, CURLOPT_PROXY, proxy.c_str ());
curl_easy_setopt (easyhandle, CURLOPT_WRITEFUNCTION, WriteCallback);
curl_easy_setopt (easyhandle, CURLOPT_WRITEDATA, &readBuffer);
curl_mime *multipart = curl_mime_init (easyhandle);
curl_mimepart *part = nullptr;
switch (jirCmd) {
case SEND: {
LOG ("SEND: " << (urlBase+uploadPage));
curl_easy_setopt (easyhandle, CURLOPT_URL, (urlBase+uploadPage).c_str ());
LOG ("maxUploadSize: " << maxUploadSize);
long uploadsize = (size_t) maxUploadSize;
curl_easy_setopt (easyhandle, CURLOPT_INFILESIZE, uploadsize);
LOG ("time: " << minimumAvailability);
part = curl_mime_addpart (multipart);
curl_mime_name (part, "time");
curl_mime_data (part, minimumAvailability.c_str (), CURL_ZERO_TERMINATED);
if (password.size ()) {
LOG ("key: " << password);
part = curl_mime_addpart (multipart);
curl_mime_name (part, "key");
curl_mime_data (part, password.c_str (), CURL_ZERO_TERMINATED);
}
LOG ("inputFileName: " << bfs::path (inputFileName).filename ());
part = curl_mime_addpart (multipart);
curl_mime_name (part, "file");
if (contentType.length ()) {
LOG ("contentType: " << contentType);
curl_mime_type (part, contentType.c_str ());
}
if (attachName.empty ()) {
attachName = bfs::path (inputFileName).filename ().c_str ();
LOG ("attachName: " << attachName);
}
curl_mime_filename (part, attachName.c_str ());
FILE *fp = fopen (inputFileName.c_str (), "r");
fseek (fp, 0L, SEEK_END);
long int fsize (ftell (fp));
fseek (fp, 0L, SEEK_SET);
curl_mime_data_cb (part, fsize,
(curl_read_callback) fread,
(curl_seek_callback) fseek,
NULL, //(curl_seek_callback) fclose,
fp);
}
break;
case UPDATE: {
LOG ("UPDATE: " << (urlBase+updatePage));
curl_easy_setopt (easyhandle, CURLOPT_URL, (urlBase+updatePage).c_str ());
LOG ("h: " << inputFileName);
part = curl_mime_addpart (multipart);
curl_mime_name (part, "h");
curl_mime_data (part, inputFileName.c_str (), CURL_ZERO_TERMINATED);
LOG ("u: " << minimumAvailability);
part = curl_mime_addpart (multipart);
curl_mime_name (part, "u");
curl_mime_data (part, minimumAvailability.c_str (), CURL_ZERO_TERMINATED);
}
break;
}
curl_easy_setopt (easyhandle, CURLOPT_MIMEPOST, multipart);
CURLcode res (curl_easy_perform (easyhandle));
curl_easy_cleanup (easyhandle);
cout << readBuffer << endl;
showTime ("Upload");
if (res != CURLE_OK)
cerr << prog << " failed: " << curl_easy_strerror (res) << endl;
return 0;
}
// ================================================================================

View File

@ -65,6 +65,14 @@ const string kaz::availableURLChars =
"abcdefghijklmnopqrstuvwxyz" "abcdefghijklmnopqrstuvwxyz"
"~"; "~";
const regex kaz::encodedWordRegex ("\\s*=\\?" // flag begin
"([0-9A-Za-z!#$%&'+^_`{}~-]+)" // charset
"\\?" // flag sep
"([QqBb])" // quoted our base64
"\\?" // flag sep
"([^ ?]+)" // encoded string
"\\?=\\s*"); // flag end
// ================================================================================ // ================================================================================
uint16_t uint16_t
@ -97,7 +105,7 @@ kaz::ns2string (const double &delta) {
// ================================================================================ // ================================================================================
void void
kaz::replaceAll (string& str, const string &from, const string &to) { kaz::replaceAll (string& str, const string &from, const string &to) {
DEF_LOG ("kaz::replaceAll", "form: " << from << " to: " << to); DEF_LOG ("kazMisc::replaceAll", "form: " << from << " to: " << to);
if (str.empty () || from.empty ()) if (str.empty () || from.empty ())
return; return;
for (string::size_type startPos (0); for (string::size_type startPos (0);
@ -108,7 +116,7 @@ kaz::replaceAll (string& str, const string &from, const string &to) {
void void
kaz::replaceAll (string& str, const map<const string, const string> &subst) { kaz::replaceAll (string& str, const map<const string, const string> &subst) {
DEF_LOG ("kaz::replaceAll", "str: " << str); DEF_LOG ("kazMisc::replaceAll", "str: " << str.substr (0, 100) << "...");
for (map<const string, const string>::const_iterator it = subst.begin (); it != subst.end (); ++it) for (map<const string, const string>::const_iterator it = subst.begin (); it != subst.end (); ++it)
replaceAll (str, it->first, it->second); replaceAll (str, it->first, it->second);
} }
@ -116,16 +124,16 @@ kaz::replaceAll (string& str, const map<const string, const string> &subst) {
// ================================================================================ // ================================================================================
void void
kaz::toLower (string &content) { kaz::toLower (string &content) {
DEF_LOG ("kaz::toLower", "content: " << content); DEF_LOG ("kazMisc::toLower", "content: " << content.substr (0, 100) << "...");
static locale loc; static locale loc;
for (string::size_type i = 0; i < content.length (); ++i) for (string::size_type i = 0; i < content.length (); ++i)
content [i] = tolower (content[i], loc); content [i] = tolower (content[i], loc);
LOG ("content: " << content); LOG ("content: " << content.substr (0, 100) << "...");
} }
const string & const string &
kaz::toUpperIfNeed (const string &src, string &tmp) { kaz::toUpperIfNeed (const string &src, string &tmp) {
DEF_LOG ("kaz::toUpperIfNeed", "src: " << src); DEF_LOG ("kazMisc::toUpperIfNeed", "src: " << src);
for (string::const_iterator it = src.begin (); it != src.end (); ++it) for (string::const_iterator it = src.begin (); it != src.end (); ++it)
if (*it != toupper (*it)) { if (*it != toupper (*it)) {
tmp.reserve (); tmp.reserve ();
@ -143,7 +151,7 @@ caseInsensitiveCharCompare (char a, char b) {
string::size_type string::size_type
kaz::caseInsensitiveFind (const string& s, const string& pattern, const string::size_type &pos) { kaz::caseInsensitiveFind (const string& s, const string& pattern, const string::size_type &pos) {
DEF_LOG ("kaz::caseInsensitiveFind", "pattern: " << pattern << " pos: " << pos << " s: " << s); DEF_LOG ("kazMisc::caseInsensitiveFind", "pattern: " << pattern << " pos: " << pos << " s: " << s.substr (0, 100) << "...");
string tmp; string tmp;
const string &upperPattern (toUpperIfNeed (pattern, tmp)); const string &upperPattern (toUpperIfNeed (pattern, tmp));
LOG ("pattern: " << upperPattern); LOG ("pattern: " << upperPattern);
@ -156,7 +164,7 @@ kaz::caseInsensitiveFind (const string& s, const string& pattern, const string::
string::size_type string::size_type
kaz::caseInsensitiveRFind (const string& s, const string& pattern, const string::size_type &pos) { kaz::caseInsensitiveRFind (const string& s, const string& pattern, const string::size_type &pos) {
DEF_LOG ("kaz::caseInsensitiveRFind", "pattern: " << pattern << " pos: " << pos << " s: " << s); DEF_LOG ("kazMisc::caseInsensitiveRFind", "pattern: " << pattern << " pos: " << pos << " s: " << s.substr (0, 100) << "...");
string tmp; string tmp;
const string &upperPattern (toUpperIfNeed (pattern, tmp)); const string &upperPattern (toUpperIfNeed (pattern, tmp));
LOG ("pattern: " << upperPattern); LOG ("pattern: " << upperPattern);
@ -167,11 +175,24 @@ kaz::caseInsensitiveRFind (const string& s, const string& pattern, const string:
return s.rend () - it - pattern.length (); return s.rend () - it - pattern.length ();
} }
string
kaz::boundaryGen (const int &size) {
static const char alphanum[] =
"0123456789"
"ABCDEFGHIJKLMNOPQRSTUVWXYZ"
"abcdefghijklmnopqrstuvwxyz";
string result;
result.reserve (size);
for (int i = 0; i < size; ++i)
result += alphanum[rand() % (sizeof (alphanum) - 1)];
return result;
}
// ================================================================================ // ================================================================================
template<char delim> template<char delim>
void void
kaz::quotedDecode (string &content) { kaz::quotedDecode (string &content) {
DEF_LOG ("kaz::quotedDecode", "delim: " << delim << " content: " << content); DEF_LOG ("kazMisc::quotedDecode", "delim: " << delim << " content: " << content.substr (0, 100) << "...");
string::size_type len (content.length ()); string::size_type len (content.length ());
if (!len) if (!len)
return; return;
@ -185,25 +206,24 @@ kaz::quotedDecode (string &content) {
continue; continue;
} }
if (p+1 < content.end () && *(p+1) == '\n') { if (p+1 < content.end () && *(p+1) == '\n') {
LOG_BUG (q == content.begin (), ++p;continue, "kazMisc::quotedDecode bug: bad quoted-printable format. (start with '=', delim: " << int (delim) << " content: " << content.substr (0, 100) << "...)");
LOG_BUG (q == content.begin (), ++p;continue, "kazMisc::quotedDecode bug: bad quoted-printable format. (start with '=', content: " << content << ")");
++p; ++p;
--q; --q;
continue; continue;
} }
LOG_BUG (p+3 > content.end () || !isxdigit (p[1]) || !isxdigit (p[2]), return, "kazMisc::quotedDecode bug: bad quoted-printable format. (content: " << content << ")"); LOG_BUG (p+3 > content.end () || !isxdigit (p[1]) || !isxdigit (p[2]), return, "kazMisc::quotedDecode bug: bad quoted-printable format. (delim: " << int (delim) << " p:" << content.substr (p-content.begin (), 3) << " content: " << content.substr (0, 100) << "... len: " << len << ")");
*q = (char) ((getHexaVal (p[1]) << 4) + getHexaVal (p[2])); *q = (char) ((getHexaVal (p[1]) << 4) + getHexaVal (p[2]));
p += 2; p += 2;
} }
content.resize (q-content.begin ()); content.resize (q-content.begin ());
LOG ("content: " << content); LOG ("content: " << content.substr (0, 100) << "...");
} }
// ================================================================================ // ================================================================================
void void
kaz::quotedEncode (string &content) { kaz::quotedEncode (string &content) {
DEF_LOG ("kaz::quotedDecode", "content: " << content); DEF_LOG ("kazMisc::quotedDecode", "content: " << content.substr (0, 100) << "...");
string::size_type nbQuoted (0); string::size_type nbQuoted (0);
for (string::const_iterator it = content.begin (); it != content.end (); ++it) for (string::const_iterator it = content.begin (); it != content.end (); ++it)
if (isQuotedPrintable (*it)) if (isQuotedPrintable (*it))
@ -246,30 +266,36 @@ kaz::quotedEncode (string &content) {
++cols; ++cols;
} }
content.swap (result); content.swap (result);
LOG ("content: " << content); LOG ("content: " << content.substr (0, 100) << "...");
} }
// ================================================================================ // ================================================================================
void void
kaz::base64Decode (string &content) { kaz::base64Decode (string &content) {
DEF_LOG ("kaz::base64Decode", "content: " << content); DEF_LOG ("kazMisc::base64Decode", "content: " << content.substr (0, 100) << "...");
string::size_type len (content.length ()); string::size_type len (content.length ());
if (!len) if (!len)
return; return;
LOG ("len: " << len); LOG ("len: " << len);
unsigned char buff[4]; unsigned char buff[4];
int idx = 0; int idx = 0;
string::iterator p (content.begin ()), q (p); string::iterator p (content.begin ()), q (p), lastOK (p);
for (; for (;
p < content.end (); p < content.end ();
++p) { ++p) {
char c = *p; char c = *p;
if (c == '=') if (c == '=')
break; break;
if (c == '\n') if (c == '\n') {
lastOK = p;
continue; continue;
}
LOG_BUG (!isBase64 (c), return, "kazMisc::base64Decode bug: bad base64 format. (content: " << content << ")"); if (!isBase64 (c)) {
content.resize (lastOK-content.begin ());
LOG ("kazMisc::base64Decode bug: bad base64 format. (content: " << content.substr (0, 100) << "...)");
return;
}
buff [idx] = getBase64Val (c); buff [idx] = getBase64Val (c);
if (++idx != 4) if (++idx != 4)
continue; continue;
@ -284,20 +310,19 @@ kaz::base64Decode (string &content) {
buff [j] = 0; buff [j] = 0;
*q = buff [0] << 2 | (buff [1] & 0x30) >> 4; *q = buff [0] << 2 | (buff [1] & 0x30) >> 4;
++q; ++q;
--idx; if (idx > 2) {
if (idx) {
*q = buff [1] << 4 | (buff [2] & 0x3c) >> 2; *q = buff [1] << 4 | (buff [2] & 0x3c) >> 2;
++q; ++q;
} }
} }
content.resize (q-content.begin ()); content.resize (q-content.begin ());
LOG ("content: " << content); LOG ("content: " << content.substr (0, 100) << "...");
} }
// ================================================================================ // ================================================================================
void void
kaz::base64Encode (string &content) { kaz::base64Encode (string &content) {
DEF_LOG ("kaz::base64Encode", "content: " << content); DEF_LOG ("kazMisc::base64Encode", "content: " << content.substr (0, 100) << "...");
string::size_type length (content.length ()); string::size_type length (content.length ());
std::string result; std::string result;
result.reserve ((length + 2) / 3 * 4 + length / MAX_QUOTED_PRINTABLE_SIZE + 1); result.reserve ((length + 2) / 3 * 4 + length / MAX_QUOTED_PRINTABLE_SIZE + 1);
@ -327,13 +352,13 @@ kaz::base64Encode (string &content) {
} }
} }
content = result; content = result;
LOG ("content: " << content); LOG ("content: " << content.substr (0, 100) << "...");
} }
// ================================================================================ // ================================================================================
void void
kaz::iso2utf (string &content) { kaz::iso2utf (string &content) {
DEF_LOG ("kaz::iso2utf", "content: " << content); DEF_LOG ("kazMisc::iso2utf", "content: " << content.substr (0, 100) << "...");
string::size_type len (content.length ()); string::size_type len (content.length ());
if (!len) if (!len)
return; return;
@ -359,79 +384,97 @@ kaz::iso2utf (string &content) {
if (p == q) if (p == q)
break; break;
} }
LOG ("content: " << content); LOG ("content: " << content.substr (0, 100) << "...");
} }
// ================================================================================ // ================================================================================
void void
kaz::encodedWord (string &content) { kaz::encodedWordDecode (string &content) {
// rfc2047 // rfc2047
DEF_LOG ("kaz::extendedWord", "content: " << content); DEF_LOG ("kazMisc::encodedWordDecode", "content: " << content.substr (0, 100) << "...");
string::size_type charsetPos = content.find ("=?"); string::size_type charsetPos = content.find ("=?");
if (charsetPos == string::npos) if (charsetPos == string::npos)
return; return;
LOG ("charsetPos: " << charsetPos); LOG ("charsetPos: " << charsetPos);
LOG_BUG (charsetPos != 0, return, "kazMisc::extendedWord bug: =? not at begin pos. (content: " << content << ")");
string result; string result;
for ( ; auto pos (0);
(charsetPos = content.find ("=?", charsetPos)) != string::npos; sregex_iterator ewItEnd;
) { for (sregex_iterator ewIt (content.begin (), content.end (), encodedWordRegex);
string::size_type modePos = content.find ("?", charsetPos+2); ewIt != ewItEnd;
++ewIt) {
smatch m = *ewIt;
if (pos != m.position ()) {
result += content.substr (pos, m.position () - pos);
LOG ("stantad " << content.substr (pos, m.position () - pos));
}
string encoded (m[3]);
replace (encoded.begin (), encoded.end (), '_', ' ');
LOG_BUG (modePos == string::npos, return, "kazMisc::extendedWord bug: no end chartset. (content: " << content << ")"); LOG ("charset: " << m[1] << " mode: " << m[2] << " string: " << encoded);
string::size_type contentPos = content.find ("?", modePos+1);
LOG_BUG (contentPos != modePos+2, return, "kazMisc::extendedWord bug: no end chartset. (content: " << content << ")"); switch (m[2].str ()[0]) {
string::size_type endPos = content.find ("?=", contentPos+1);
LOG_BUG (endPos == string::npos, return, "kazMisc::extendedWord bug: no end chartset. (content: " << content << ")");
string tmp (content.substr (contentPos+1, endPos-contentPos-1));
switch (content [modePos+1]) {
case 'B': case 'B':
case 'b': case 'b':
base64Decode (tmp); base64Decode (encoded);
break; break;
case 'Q': case 'Q':
case 'q': case 'q':
quotedDecode (tmp); quotedDecode (encoded);
break; break;
default: default:
LOG_BUG (true, return, "kazMisc::extendedWord bug: unknown mode. (mode: " << content [modePos+1] << ")"); LOG_BUG (true, return, "kazMisc::encodedWordDecode bug: unknown mode. (mode: " << m[2] << ")");
} }
LOG ("tmp: " << tmp); LOG ("decoded: " << encoded);
string charset (content.substr (charsetPos, modePos-charsetPos-2)); string charset (m[1]);
toLower (charset); toLower (charset);
if (! caseInsensitiveFind (charset, "ISO")) if (! caseInsensitiveFind (charset, "ISO"))
iso2utf (tmp); iso2utf (encoded);
result += tmp; result += encoded;
charsetPos = endPos+2; pos = m.position () + m.str ().length ();
} }
content = result; content = result + content.substr (pos);
LOG ("content: " << content); LOG ("content: " << content.substr (0, 100) << "...");
} }
// ================================================================================ // ================================================================================
void void
kaz::charsetValue (string &content) { kaz::charsetValueDecode (string &content) {
// rfc2184 // rfc2184
DEF_LOG ("kaz::charsetValue", "content: " << content); DEF_LOG ("kazMisc::charsetValueDecode", "content: " << content.substr (0, 100) << "...");
string::size_type langPos = content.find ("'"); string::size_type langPos = content.find ("'");
LOG_BUG (langPos == string::npos, return, "kazMisc::charsetValue bug: no '. (content: " << content << ")"); LOG_BUG (langPos == string::npos, return, "kazMisc::charsetValueDecode bug: no '. (content: " << content.substr (0, 100) << "...)");
string::size_type contentPos = content.find ("'", langPos+1); string::size_type contentPos = content.find ("'", langPos+1);
LOG_BUG (contentPos == string::npos, return, "kazMisc::charsetValue bug: no double '. (content: " << content << ")"); LOG_BUG (contentPos == string::npos, return, "kazMisc::charsetValueDecode bug: no double '. (content: " << content.substr (0, 100) << "...)");
string tmp (content.substr (contentPos+1)); string tmp (content.substr (contentPos+1));
quotedDecode<'%'> (tmp); quotedDecode<'%'> (tmp);
LOG ("tmp: " << tmp); LOG ("tmp: " << tmp.substr (0, 100) << "...");
string charset (content.substr (0, langPos)); string charset (content.substr (0, langPos));
toLower (charset); toLower (charset);
if (! caseInsensitiveFind (charset, "ISO")) if (! caseInsensitiveFind (charset, "ISO"))
iso2utf (tmp); iso2utf (tmp);
content = tmp; content = tmp;
LOG ("content: " << content); LOG ("content: " << content.substr (0, 100) << "...");
}
// ================================================================================
void
kaz::removeQuote (string &content) {
if (content.empty () || content [0] != '"')
return;
string::size_type stop = (1);
for (;;) {
stop = content.find ('"', stop);
if (stop == string::npos || content [stop-1] != '\\')
break;
++stop;
}
content = (stop != string::npos) ?
content.substr (1, stop-1) :
content.substr (1);
} }
// ================================================================================ // ================================================================================

223
src/cpp/server.cpp Normal file
View File

@ -0,0 +1,223 @@
////////////////////////////////////////////////////////////////////////////
// Copyright KAZ 2021 //
// //
// contact (at) kaz.bzh //
// //
// This software is a filter to shrink email by attachment extraction. //
// //
// This software is governed by the CeCILL-B license under French law and //
// abiding by the rules of distribution of free software. You can use, //
// modify and/or redistribute the software under the terms of the //
// CeCILL-B license as circulated by CEA, CNRS and INRIA at the following //
// URL "http://www.cecill.info". //
// //
// As a counterpart to the access to the source code and rights to copy, //
// modify and redistribute granted by the license, users are provided //
// only with a limited warranty and the software's author, the holder of //
// the economic rights, and the successive licensors have only limited //
// liability. //
// //
// In this respect, the user's attention is drawn to the risks associated //
// with loading, using, modifying and/or developing or reproducing the //
// software by the user in light of its specific status of free software, //
// that may mean that it is complicated to manipulate, and that also //
// therefore means that it is reserved for developers and experienced //
// professionals having in-depth computer knowledge. Users are therefore //
// encouraged to load and test the software's suitability as regards //
// their requirements in conditions enabling the security of their //
// systems and/or data to be ensured and, more generally, to use and //
// operate it in the same conditions as regards security. //
// //
// The fact that you are presently reading this means that you have had //
// knowledge of the CeCILL-B license and that you accept its terms. //
////////////////////////////////////////////////////////////////////////////
#include <boost/program_options.hpp>
#include <fcntl.h>
#include <map>
#include <netinet/in.h>
#include <poll.h>
#include <stdio.h>
#include <stdlib.h>
#include <string>
#include <sys/socket.h>
#include <unistd.h>
#include <vector>
#include "kazDebug.hpp"
#include "kazMisc.hpp"
using namespace std;
using namespace boost;
using namespace boost::program_options;
using namespace kaz;
#define CONSOLE(expr) {if (!quietFlag) {std::cerr << Log::getLocalTimeStr () << " " << expr << std::endl << std::flush; }}
// ================================================================================
const string LAST_VERSION_NUM ("3.0");
const string LAST_VERSION_DATE ("2024-01-01");
const string LAST_VERSION (LAST_VERSION_NUM+" "+LAST_VERSION_DATE+" server");
#define PORT 8080
const string BASH ("/bin/bash");
const string FILTER_CMD ("/home/filter/testCopyInOut.sh");
// ================================================================================
static options_description mainDescription ("Main options", getCols ());
static options_description hide ("Hidded options", getCols ());
static const char *prog = NULL;
// ================================================================================
void
usage (const string &msg = "", const bool &hidden = false) {
if (!msg.empty ()) {
cout << msg << endl;
exit (1);
}
cout << endl
<< "Usage: " << endl
<< " " << prog << " [-p port] [-f filterFileName.sh]" << endl
<< endl << mainDescription
<< endl;
if (hidden)
cout << hide << endl;
exit (0);
}
void
version () {
cout << LAST_VERSION << " KAZ team production (https://kaz.bzh/)" << endl;
exit (0);
}
// ================================================================================
int
main (int argc, const char *argv[], char **envp) {
// uncomment next line in case of debug parse options
// Log::debug = true;
DEF_LOG ("main:", LAST_VERSION);
prog = argv [0];
bool
helpFlag (false),
versionFlag (false),
quietFlag (false),
useTheForceLuke (false),
debugFlag (false);
int port (PORT);
string filterFileName (FILTER_CMD);
try {
mainDescription.add_options ()
("help,h", bool_switch (&helpFlag), "produce this help message")
("version,v", bool_switch (&versionFlag), "display version information")
("quiet,q", bool_switch (&quietFlag), "quiet mode")
("port,p", value<int> (&port)->default_value (port), "server port number")
("filter,f", value<string> (&filterFileName)->default_value (filterFileName), "filter file name script")
;
hide.add_options ()
("useTheForceLuke", bool_switch (&useTheForceLuke), "display hidded options")
("debug,g", bool_switch (&debugFlag), "debug mode")
;
options_description cmd ("All options");
cmd.add (mainDescription).add (hide).add_options ();
variables_map vm;
store (parse_command_line(argc, argv, cmd), vm);
notify (vm);
if (debugFlag) {
#ifdef DISABLE_LOG
cerr << "No debug option available (was compiled with -DDISABLE_LOG)" << endl;
#endif
}
Log::debug = debugFlag;
if (useTheForceLuke)
usage ("", true);
if (versionFlag)
version ();
if (helpFlag)
usage ();
} catch (std::exception &e) {
cerr << "error: " << e.what() << endl;
usage ();
return 1;
} catch (...) {
cerr << "Exception of unknown type!" << endl;
return 1;
}
struct sockaddr_in address;
int opt = 1;
socklen_t addrlen = sizeof (address);
LOG ("create socket");
int serverSocket (socket (AF_INET, SOCK_STREAM, 0));
if (serverSocket < 0) {
perror ("socket failed");
exit (EXIT_FAILURE);
}
LOG ("set socket options");
if (setsockopt (serverSocket, SOL_SOCKET,
SO_REUSEADDR | SO_REUSEPORT, &opt,
sizeof (opt))) {
perror ("setsockopt");
exit (EXIT_FAILURE);
}
address.sin_family = AF_INET;
address.sin_addr.s_addr = INADDR_ANY;
address.sin_port = htons (port);
LOG ("bind");
if (bind (serverSocket, (struct sockaddr*)&address, sizeof (address)) < 0) {
perror ("bind failed");
exit (EXIT_FAILURE);
}
LOG ("listen");
if (listen (serverSocket, 3) < 0) {
perror ("listen");
exit (EXIT_FAILURE);
}
CONSOLE ("Server started on port " << port);
for (;;) {
LOG ("accept");
int clientSocket (accept (serverSocket, (struct sockaddr*) &address, &addrlen));
if (clientSocket < 0) {
perror ("accept");
// XXX ne pas quitter
exit (EXIT_FAILURE);
}
char *command[] = {const_cast<char *> (BASH.c_str ()),
const_cast<char *> (filterFileName.c_str ()), NULL};
switch (fork ()) {
case -1:
perror ("fork");
exit (EXIT_FAILURE);
case 0:
CONSOLE ("New request");
close (STDOUT_FILENO);
close (STDIN_FILENO);
dup2 (clientSocket, STDIN_FILENO);
dup2 (clientSocket, STDOUT_FILENO);
execve (BASH.c_str (), command, envp);
perror ("execve");
break;
default:
close (clientSocket);
break;
}
}
// XXX condition de sortie ?
close (serverSocket);
return 0;
}
// ================================================================================

118
src/cpp/testServerRW.cpp Normal file
View File

@ -0,0 +1,118 @@
////////////////////////////////////////////////////////////////////////////
// Copyright KAZ 2021 //
// //
// contact (at) kaz.bzh //
// //
// This software is a filter to shrink email by attachment extraction. //
// //
// This software is governed by the CeCILL-B license under French law and //
// abiding by the rules of distribution of free software. You can use, //
// modify and/or redistribute the software under the terms of the //
// CeCILL-B license as circulated by CEA, CNRS and INRIA at the following //
// URL "http://www.cecill.info". //
// //
// As a counterpart to the access to the source code and rights to copy, //
// modify and redistribute granted by the license, users are provided //
// only with a limited warranty and the software's author, the holder of //
// the economic rights, and the successive licensors have only limited //
// liability. //
// //
// In this respect, the user's attention is drawn to the risks associated //
// with loading, using, modifying and/or developing or reproducing the //
// software by the user in light of its specific status of free software, //
// that may mean that it is complicated to manipulate, and that also //
// therefore means that it is reserved for developers and experienced //
// professionals having in-depth computer knowledge. Users are therefore //
// encouraged to load and test the software's suitability as regards //
// their requirements in conditions enabling the security of their //
// systems and/or data to be ensured and, more generally, to use and //
// operate it in the same conditions as regards security. //
// //
// The fact that you are presently reading this means that you have had //
// knowledge of the CeCILL-B license and that you accept its terms. //
////////////////////////////////////////////////////////////////////////////
#include <arpa/inet.h>
#include <iostream>
#include <stdio.h>
#include <string>
#include <sys/socket.h>
#include <unistd.h>
#include "kazDebug.hpp"
using namespace std;
using namespace kaz;
const string LAST_VERSION_NUM ("3.0");
const string LAST_VERSION_DATE ("2024-01-01");
const string LAST_VERSION (LAST_VERSION_NUM+" "+LAST_VERSION_DATE+" testServerRW");
#define PORT 8080
int
main (int argc, char const* argv[]) {
Log::debug = true;
DEF_LOG ("main:", LAST_VERSION);
int clientSocket;
struct sockaddr_in serv_addr;
LOG ("create socket");
if ((clientSocket = socket (AF_INET, SOCK_STREAM, 0)) < 0) {
cerr << endl << "Socket creation error" << endl;
perror ("socket failed");
return -1;
}
serv_addr.sin_family = AF_INET;
serv_addr.sin_port = htons (PORT);
LOG ("check address");
if (inet_pton (AF_INET, "127.0.0.1", &serv_addr.sin_addr) <= 0) {
cerr << endl << "Invalid address / Address not supported" << endl;
perror ("socket failed");
return -1;
}
LOG ("connect on " << PORT);
if (connect (clientSocket, (struct sockaddr *) &serv_addr, sizeof (serv_addr)) < 0) {
cerr << endl << "Connection Failed" << endl;
perror ("socket failed");
return -1;
}
bool end (false);
LOG ("send original email (end with QUIT)");
for (;;) {
if (!end) {
string hello;
cin >> hello;
if (hello == "QUIT") {
shutdown (clientSocket, SHUT_WR);
LOG ("shutdown send");
break;
}
LOG ("avant");
hello.push_back ('\n');
send (clientSocket, hello.c_str (), hello.size (), 0);
LOG ("apres");
}
}
LOG ("receive...");
for (;;) {
char buffer[1024] = { 0 };
ssize_t msgSize (recv (clientSocket, buffer, 1024 - 1, 0));
switch (msgSize) {
case -1 :
perror ("socket failed");
return -1;
case 0 :
LOG ("shutdown receive");
close (clientSocket);
return 0;
break;
default:
//cout.write (buffer, msgSize);
printf ("%s\n", buffer);
}
}
}

View File

@ -54,7 +54,7 @@ namespace kaz {
//static const vector<const string> stringsToUpdate; //static const vector<const string> stringsToUpdate;
static vector<string> stringsToUpdate; static vector<string> stringsToUpdate;
/*! mime tokens */ /*! mime tokens */
static const string contentTypeToken, contentDispositionToken, contentTransferEncodingToken, base64Token, quotedPrintableToken, contentIDToken, PLAIN, HTML, RELATED, ALTERNATIVE; static const string contentTypeToken, contentDispositionToken, contentTransferEncodingToken, base64Token, quotedPrintableToken, contentIDToken, PLAIN, HTML, MULTIPART, RELATED, ALTERNATIVE, SIGNED, KAZ_ATTACH_NAME;
/*! pattern to extract mime values */ /*! pattern to extract mime values */
static const regex nameRegEx, nameCharsetRegEx, boundaryRegEx, cidDefRegEx, textRegEx, multiRegEx; static const regex nameRegEx, nameCharsetRegEx, boundaryRegEx, cidDefRegEx, textRegEx, multiRegEx;
@ -100,7 +100,7 @@ namespace kaz {
/*! char position of attachment content */ /*! char position of attachment content */
streamoff contentPos, endPos; streamoff contentPos, endPos;
/*! properties of the attachment */ /*! properties of the attachment */
bool toExtract, toUpdate, toDisclaim; bool toExtract, toUpdate, toDisclaim, isKazAttachment, isSigned;
/*! id of an image embedded in mbox */ /*! id of an image embedded in mbox */
string cid; string cid;
/*! url to replace the attachment */ /*! url to replace the attachment */
@ -131,7 +131,9 @@ namespace kaz {
/*! recursively marks alternative attachments to be disclaim */ /*! recursively marks alternative attachments to be disclaim */
void markDisclaim (bool &plainMarked, bool &htmlMarked); void markDisclaim (bool &plainMarked, bool &htmlMarked);
/*! recursively marks big attachments to be removed and upated (including disclaim). return true when part need to be updated (can't be extracted). */ /*! recursively marks big attachments to be removed and upated (including disclaim). return true when part need to be updated (can't be extracted). */
bool markSignificant (const string &parentMultiProp, const streamoff &minAttachSize, ifstream &mbox, vector<Attachment *> &allMarkedPtrs); bool markSignificant (const string &parentMultiProp, const bool &parentSigned, const streamoff &minAttachSize, ifstream &mbox, vector<Attachment *> &allMarkedPtrs);
/*! get a copy of mime header */
string getMime (ifstream &mbox) const;
/*! get a copy of the content. Base64 is decoded. Quoted-Printable is unwarp and unquoted */ /*! get a copy of the content. Base64 is decoded. Quoted-Printable is unwarp and unquoted */
string getContent (ifstream &mbox) const; string getContent (ifstream &mbox) const;
/*! write the content, encoded if necessary (base64 and quoted-printable) */ /*! write the content, encoded if necessary (base64 and quoted-printable) */

View File

@ -44,6 +44,19 @@ namespace kaz {
namespace bfs = boost::filesystem; namespace bfs = boost::filesystem;
// ================================================================================ // ================================================================================
/*! place to add download link (footer, attachment or both) */
enum AttachMode { NONE = 0, FOOTER = 1, ATTACHMENT = 2, BOTH = (FOOTER|ATTACHMENT) };
extern const string attachModeLabels[];
extern const map<string, AttachMode> attachModeMap;
ostream &operator << (ostream &out, const AttachMode &attachMode);
istream &operator >> (istream &in, AttachMode &attachMode);
enum HeaderType { SAME, MIXED, MAIN_PLAIN };
extern const string headerTypeLabels[];
extern const map<string, HeaderType> headerTypeMap;
ostream &operator << (ostream &out, const HeaderType &headerType);
istream &operator >> (istream &in, HeaderType &headerType);
/*! root level of e-mail structure */ /*! root level of e-mail structure */
class MainAttachment : public Attachment { class MainAttachment : public Attachment {
public: public:
@ -83,13 +96,23 @@ namespace kaz {
bfs::path extractDir; bfs::path extractDir;
/*! URL for download archives */ /*! URL for download archives */
string archiveDownloadURL; string archiveDownloadURL;
/*! no main text in email can be use to add disclaim */ /*! if no main text in email can be used to add disclaim */
bool forceMainText; bool emptyEMail;
/*! if contain previous kaz attachment */
bool previousKazAttachment;
/*! subset in the tree of all attachments to be consider for extraction or modification */ /*! subset in the tree of all attachments to be consider for extraction or modification */
vector<Attachment *> allMarkedPtrs; vector<Attachment *> allMarkedPtrs;
/*! previous links find in mbox */ /*! previous links find in mbox */
map<string, string> previousLinks; map<string, string> previousLinks;
/*! boundary if a mixed section is added including previous and next "--" */
string addedBoundary;
/*! size of added boundary before the last "--" */
streamoff addedBoundaryMiddleSize;
/*! moved conten-type headers */
string movedContentType;
/*! add link only if no significant value already exist. Trust the values from html.*/ /*! add link only if no significant value already exist. Trust the values from html.*/
void addPrevious (const string &href, const string &name, const bool &trust = false); void addPrevious (const string &href, const string &name, const bool &trust = false);
@ -97,11 +120,16 @@ namespace kaz {
void extractLinks (const string &extractedPlainKAZ); void extractLinks (const string &extractedPlainKAZ);
/*! extract previous links from html-li list. Used by extractPreviousKAZ */ /*! extract previous links from html-li list. Used by extractPreviousKAZ */
void extractLinks (const vector<string> &liOne); void extractLinks (const vector<string> &liOne);
/*! extract previous links in mbox on one attachment section. Used by extractPreviousKAZ */
void extractPreviousKAZ (string &extractedPlainKAZ, string &extractedHtmlKAZ, ifstream &mbox, const Attachment &attach);
/*! extract previous links in mbox. Used by getUpdatedURL and substitute */ /*! extract previous links in mbox. Used by getUpdatedURL and substitute */
void extractPreviousKAZ (ifstream &mbox); void extractPreviousKAZ (ifstream &mbox);
/*! remove previous links to archive. Used by substitute */ /*! remove previous links to archive. Used by substitute */
void removePreviousArchive (); void removePreviousArchive ();
/*! rewrite main headers */
void rewriteHeaders (ifstream &mbox, ofstream &outbox, const HeaderType &headerType);
public: public:
/*! the main attachment in mbox */ /*! the main attachment in mbox */
MainAttachment (ifstream &mbox); MainAttachment (ifstream &mbox);
@ -115,7 +143,7 @@ namespace kaz {
/*! extract big attachments in mbox to extractDir and write to stdout le dirname of each extraction */ /*! extract big attachments in mbox to extractDir and write to stdout le dirname of each extraction */
void extract (ifstream &mbox, const SizeArg &minSize) const; void extract (ifstream &mbox, const SizeArg &minSize) const;
/*! substitute big attachments by the url give in stdin */ /*! substitute big attachments by the url give in stdin */
void substitute (ifstream &mbox, ofstream &outbox, const SizeArg &minSize); void substitute (ifstream &mbox, ofstream &outbox, const SizeArg &minSize, AttachMode attachMode);
}; };
// ================================================================================ // ================================================================================

View File

@ -38,6 +38,7 @@
#include <string> #include <string>
#include <ctype.h> #include <ctype.h>
#include <map> #include <map>
#include <regex>
namespace kaz { namespace kaz {
using namespace std; using namespace std;
@ -47,6 +48,8 @@ namespace kaz {
extern const char * const base64Chars; extern const char * const base64Chars;
/*! set of chars available in URL */ /*! set of chars available in URL */
extern const string availableURLChars; extern const string availableURLChars;
/*! pattern for encoded words */
extern const regex encodedWordRegex;
// ======================================================================= // =======================================================================
/*! get the width of the terminal */ /*! get the width of the terminal */
@ -71,6 +74,9 @@ namespace kaz {
string::size_type caseInsensitiveFind (const string& s, const string& p, const string::size_type &pos = 0); string::size_type caseInsensitiveFind (const string& s, const string& p, const string::size_type &pos = 0);
/*! reverse find upper case of p in upper case of s */ /*! reverse find upper case of p in upper case of s */
string::size_type caseInsensitiveRFind (const string& s, const string& p, const string::size_type &pos = 0); string::size_type caseInsensitiveRFind (const string& s, const string& p, const string::size_type &pos = 0);
string boundaryGen (const int &size);
/*! side effect to repplace =XX by the char with de haxe value XX. It could be %XX in rfc2184 */ /*! side effect to repplace =XX by the char with de haxe value XX. It could be %XX in rfc2184 */
template<char delim='='> template<char delim='='>
void quotedDecode (string &content); void quotedDecode (string &content);
@ -82,17 +88,19 @@ namespace kaz {
void base64Encode (string &content); void base64Encode (string &content);
/*! side effect to change charset of content */ /*! side effect to change charset of content */
void iso2utf (string &content); void iso2utf (string &content);
/*! side effect to get the encodedWord according rfc2047 */ /*! side effect to get the encoded word according rfc2047 rfc5987 rfc2978 */
void encodedWord (string &content); void encodedWordDecode (string &content);
/*! side effect to get the charsetValue according rfc2184 */ /*! side effect to get the charsetValue according rfc2184 */
void charsetValue (string &content); void charsetValueDecode (string &content);
/*! side effect to remove quote */
void removeQuote (string &content);
// ======================================================================= // =======================================================================
/*! return if the c need no quote */ /*! return if the c need no quote */
inline bool inline bool
isQuotedPrintable (const char &c) { isQuotedPrintable (const char &c) {
return return
c == ' ' || c == '\t' || (c >= 33 && c <= 126 && c != '=' && c != '.'); c == ' ' || (c >= 33 && c <= 126 && c != '=' && c != '.');
// '.' is available in rfc2184 but it avoid to check '.' alone in a line :-) // '.' is available in rfc2184 but it avoid to check '.' alone in a line :-)
} }

48
src/include/version.hpp Normal file
View File

@ -0,0 +1,48 @@
////////////////////////////////////////////////////////////////////////////
// Copyright KAZ 2021 //
// //
// contact (at) kaz.bzh //
// //
// This software is a filter to shrink email by attachment extraction. //
// //
// This software is governed by the CeCILL-B license under French law and //
// abiding by the rules of distribution of free software. You can use, //
// modify and/or redistribute the software under the terms of the //
// CeCILL-B license as circulated by CEA, CNRS and INRIA at the following //
// URL "http://www.cecill.info". //
// //
// As a counterpart to the access to the source code and rights to copy, //
// modify and redistribute granted by the license, users are provided //
// only with a limited warranty and the software's author, the holder of //
// the economic rights, and the successive licensors have only limited //
// liability. //
// //
// In this respect, the user's attention is drawn to the risks associated //
// with loading, using, modifying and/or developing or reproducing the //
// software by the user in light of its specific status of free software, //
// that may mean that it is complicated to manipulate, and that also //
// therefore means that it is reserved for developers and experienced //
// professionals having in-depth computer knowledge. Users are therefore //
// encouraged to load and test the software's suitability as regards //
// their requirements in conditions enabling the security of their //
// systems and/or data to be ensured and, more generally, to use and //
// operate it in the same conditions as regards security. //
// //
// The fact that you are presently reading this means that you have had //
// knowledge of the CeCILL-B license and that you accept its terms. //
////////////////////////////////////////////////////////////////////////////
#ifndef _kaz_version_hpp
#define _kaz_version_hpp
#include <string>
namespace kaz {
// ================================================================================
extern const std::string LAST_VERSION_NUM, LAST_VERSION_DATE, LAST_VERSION;
// ================================================================================
}
#endif // _kaz_version_hpp

25
src/mainpage.md Normal file
View File

@ -0,0 +1,25 @@
**Dépollueur**
Le dépollueur consiste à extraire les pièces jointes pour les remplacer par des liens de téléchargement temporaire.
Le logiciel est donc très fortement lié à un logiciel de dépôt de fichier.
Le système de dépôt de proposé par Jirafeau à l'avantage de gérer les liens interne en fonction de la signature du contenu (hash).
Cela offre plusieurs avantages. les fichiers de même contenu sont factorisés sans que les utilisateurs en aient consciences.
L'intégrité des données est inhérente à ce mécanisme, puisque un lien ne va pas faire référence à un autre contenu dans le dépôt.
A partir du moment où notre dépollueur à une action sur les messages, cela à des conséquences sur lutilisation d'outils cryptographiques.
Le chiffrement de pièces jointes a peu d'incidence. L'information chiffrée se trouve stocké sur un serveur au lieu d'un ordinateur personnel (voir pire un serveur des GAFAM).
Ici, l'avantage est que l'information sera détruite au bout d'un temps spécifié.
La signature, empêche le dépollueur d'intervenir.
Dans le cas où l'on souhaite signer ses messages, il faut au préalable déposer ses pièces jointes (hors GAFAM) et faire référence dans le message aux liens de téléchargement.
Main Programme:
* [eMailShrinker](eMailShrinker_8cpp.html)
Main classes:
* [MainAttachment](classkaz_1_1MainAttachment.html)
* [Attachment](classkaz_1_1Attachment.html)

BIN
structures.odt Normal file

Binary file not shown.