Browse Source

Sync master branches

nikita-p 4 months ago
parent
commit
0a07493012
14 changed files with 678 additions and 760 deletions
  1. 4 1
      .gitignore
  2. 12 62
      README.md
  3. 5 0
      docs/README.md
  4. 76 0
      src/avg/README.md
  5. 8 0
      src/page/.htaccess
  6. 30 4
      src/page/README.md
  7. 88 0
      src/page/commonfoos.php
  8. 35 0
      src/page/index.php
  9. 176 175
      src/page/init.php
  10. 73 51
      src/page/main.css
  11. 0 350
      src/page/plots.php
  12. 127 117
      src/page/summary.php
  13. 7 0
      src/scripts/README.md
  14. 37 0
      src/scripts/compton_runs_script.sh

+ 4 - 1
.gitignore

@@ -1,2 +1,5 @@
 __pycache__
-*.ini
+.htaccess
+*.ini
+plotly*
+fmin.min.js

+ 12 - 62
README.md

@@ -1,68 +1,18 @@
-В этой папке лежит исходный код скриптов
+# sources
 
-### `compton_filter.py`
-код для фильтрации данных комптоновских измерений из slowdb и переноса их в базу данных калибровок
+Репозиторий с исходным кодом различных скриптов, используемых в пайплайне усреднения комптоновских энергий,
+а также некоторой документацией.
 
-#### Подробности
-Беру данные из slowdb, соответствующие измерению комптоновских энергий (`g_id==43`), очищаю (`dt > 0`), вычисляю времена начала и окончания измерения.
-Дополнительно убираю перекрывающиеся измерения (оставляю только самые новые).
-Записываю оставшиеся строки в базу данных калибровок в **Compton_run**
+## Общее описание пайплайна
+Пайплайн организован следующим образом: 
+на `cron` поставлен запуск каждые 6 часов скрипта [`compton_runs_script.sh`](./src/scripts/compton_runs_script.sh), 
+который:
+1. проводит фильтрацию измерений комптоновских энергий с помощью [`compton_filter.py`](./src/avg/compton_filter.py)
+2. усредняет измерения и записывает результаты в базу данных калибровок и в репозиторий [**tables**](https://cmd.inp.nsk.su/~compton/gogs/compton/tables), используя [`compton_combiner.py`](./src/avg/compton_combiner.py)
 
-#### Пример использования
+Для хранения информации в данном проекте и отображения состояния гит-репозиториев используется простой гит-сервер [Gogs](https://gogs.io), который доступен по ссылке [cmd.inp.nsk.su/~compton](https://cmd.inp.nsk.su/~compton)
 
-`python compton_filter.py --season cmd3_2021_2 --config database.ini --update`
+Графики с усреднениями для каждой энергетической точки и некоторая информация по сезонам отображается на [cmd.inp.nsk.su/~compton/pictures](https://cmd.inp.nsk.su/~compton/pictures)
 
-* обозначает отфильтровать таблицу **cmd3\_2021\_2** из slowdb и обновить новыми 
-значениями (если есть) базу данных калибровок
-* **database.ini** файл содержит параметры аутентификации для slowdb и бд калибровок, пример
-```
-[postgresql]
-host=xxx
-database=xxx
-user=xxx
-password=xxx
+Более подробную информацию о предназначении конкретных файлов можно получить в `README.md` в соответствующих файлам директориях.
 
-[clbrDB]
-host=xxx
-database=xxx
-user=xxx
-password=xxx
-```
-
-* требуется `python3` и библиотека `psycopg2==2.9.1` для работы
-
-### `compton_combiner.py`
-код для объединения данных заходов и измерений комптоновской энергии из бд калибровок + вычисление усреднённых значений в точках по энергии
-
-#### Подробности
-Вычитываю раны из базы данных заходов и комптоновские измерения из базы данных калибровок.
-Суммирую светимость по заходам, соответствующим каждому комптоновскому измерению. Суммарные светимости, набранные в каждом комптоновском измерении становятся весами этих измерений.
-С помощью лайклихуда усредняю комптоновские измерения с учётом весов, соответствующих светимостям.
-
-#### Пример использования
-
-`python compton_combiner.py -s NNBAR2021 -c database.ini --csv --clbrdb --pics_folder ./pics`
-
-* означает вычислить средние энергии для сезона **NNBAR2021**, использовать конфиг файл с данными аутентификации **database.ini**, сохранить полученные результаты в csv таблицу, обновить данные в базе данных калибровок, сохранить картинки по точкам в директории **./pics**
-* в дополнение к предыдущему, **database.ini** должен содержать аутентификацию для базы данных заходов, пример
-```
-[cmdruns]
-host=xxx
-database=xxx
-user=xxx
-password=xxx
-```
-
-### `compton_checker.py`
-Служебный скрипт для проверки корректности записанных усреднений в базе данных калибровок
-
-### `elabelizer.py`
-Скрипт для сопоставления `elabel` и усреднённых энергий
-
-#### Пример использования
-1. Зайти на cc-8
-`ssh username@slXXcmd -p 1022`
-2. Настроить окружение
-`source /sl/cmd3/cc8-64/Cmd3Off/tune.cmd3_runs_scripts.sh`
-3. Запустить скрипт
-`python3 elabelizer.py`

+ 5 - 0
docs/README.md

@@ -0,0 +1,5 @@
+В этой папке содержится информация о методе усреднения
+
+### [`Compton.pdf`](./Compton.pdf)
+Общая информация с методикой усреднения
+

+ 76 - 0
src/avg/README.md

@@ -0,0 +1,76 @@
+Директория, содрежащая python-скрипты для пайплайна усреднения
+
+### [`compton_filter.py`](./compton_filter.py)
+код для фильтрации данных комптоновских измерений из slowdb и переноса их в базу данных калибровок
+
+#### Подробности
+Беру данные из slowdb, соответствующие измерению комптоновских энергий (`g_id==43`), очищаю (`dt > 0`), вычисляю времена начала и окончания измерения.
+Дополнительно убираю перекрывающиеся измерения (оставляю только самые новые).
+Записываю оставшиеся строки в базу данных калибровок в **Compton_run**
+
+#### Пример использования
+
+```bash
+python compton_filter.py --season cmd3_2021_2 --config database.ini --update
+```
+
+* обозначает отфильтровать таблицу **cmd3\_2021\_2** из slowdb и обновить новыми 
+значениями (если есть) базу данных калибровок
+* **database.ini** файл содержит параметры аутентификации для slowdb и бд калибровок, пример
+```ini
+[postgresql]
+host=xxx
+database=xxx
+user=xxx
+password=xxx
+
+[clbrDB]
+host=xxx
+database=xxx
+user=xxx
+password=xxx
+```
+
+* требуется `python3` и библиотека `psycopg2==2.9.1` для работы
+
+### [`compton_combiner.py`](./compton_combiner.py)
+код для объединения данных заходов и измерений комптоновской энергии из бд калибровок + вычисление усреднённых значений в точках по энергии
+
+#### Подробности
+Вычитываю раны из базы данных заходов и комптоновские измерения из базы данных калибровок.
+Суммирую светимость по заходам, соответствующим каждому комптоновскому измерению. Суммарные светимости, набранные в каждом комптоновском измерении становятся весами этих измерений.
+С помощью лайклихуда усредняю комптоновские измерения с учётом весов, соответствующих светимостям.
+
+#### Пример использования
+
+```bash
+python compton_combiner.py -s NNBAR2021 -c database.ini --csv --clbrdb --pics_folder ./pics
+```
+
+* означает вычислить средние энергии для сезона **NNBAR2021**, использовать конфиг файл с данными аутентификации **database.ini**, сохранить полученные результаты в csv таблицу, обновить данные в базе данных калибровок, сохранить картинки по точкам в директории **./pics**
+* в дополнение к предыдущему, **database.ini** должен содержать аутентификацию для базы данных заходов, пример
+```ini
+[cmdruns]
+host=xxx
+database=xxx
+user=xxx
+password=xxx
+```
+
+### [`compton_checker.py`](./compton_checker.py)
+Служебный скрипт для проверки корректности записанных усреднений в базе данных калибровок
+```bash
+python compton_checker.py -s HIGH2021 -c database.ini
+```
+* проверить корректность записей в базе данных калибровок для сезона **HIGH2021**
+
+### [`elabelizer.py`](./elabelizer.py)
+Скрипт для сопоставления `elabel` и усреднённых энергий
+
+#### Пример использования
+1. Зайти на cc-8
+`ssh username@slXXcmd -p 1022`
+2. Настроить окружение
+`source /sl/cmd3/cc8-64/Cmd3Off/tune.cmd3_runs_scripts.sh`
+3. Запустить скрипт
+`python3 elabelizer.py`

+ 8 - 0
src/page/.htaccess

@@ -0,0 +1,8 @@
+IndexIgnore *
+
+RewriteEngine Off
+
+<FilesMatch ".(htaccess|htpasswd|ini|phps|fla|psd|log|sh)$">
+Order Allow,Deny
+Deny from all
+</FilesMatch>

+ 30 - 4
src/page/README.md

@@ -1,7 +1,33 @@
 Здесь лежит исходный код для страниц с изображениями 
 [cmd.inp.nsk.su/~compton/pictures](https://cmd.inp.nsk.su/~compton/pictures/)
 
-* `init.php` - страница с детальной (интерактивной) информацией о конкретной энергетической точке
-* `summary.php` - страница с информацией в целом о сезоне
-* `plots.php` - неинтерактивная информация об усреднениях по точкам в сезоне
-* `main.css` - стили css для сайта
+* `index.php` — стартовая страница
+* `init.php` — страница с детальной (интерактивной) информацией о конкретной энергетической точке
+* `summary.php` — страница с информацией в целом о сезоне
+* `commonfoos.php` — набор используемых php-функций, который подгружается на других страницах
+* `main.css` — стили css для сайта
+
+
+### Развёртывание раздела pictures
+1. получить данные из этого репозитория 
+```bash
+mkdir pictures
+cd pictures
+git init 
+git remote add -f origin <this-repo-link>
+git sparse-checkout set "src/page"
+git pull origin master
+```
+2. прописать в корневом `.htaccess` проксирование: **pictures/\*** → **pictures/src/page/\***
+3. в папку **src/page** добавить конфигурационный файл `config.ini` с таким содержанием (`apitoken` можно получить в настройках профиля в gogs)
+```ini
+[gogs]
+apitoken = "xxxxxx"
+```
+4. скачать [plotly-latest.min.js](https://cdn.plot.ly/plotly-latest.min.js) и [fmin.min.js](https://github.com/benfred/fmin)
+и положить в папку **src/page**
+
+
+#### Полезные ссылки по отображению графиков в js
+* [4 Ways to Improve Your Plotly Graphs (towardsdatascience)](https://towardsdatascience.com/4-ways-to-improve-your-plotly-graphs-517c75947f7)
+* [Chartjs and Csv (createwithdata)](https://www.createwithdata.com/chartjs-and-csv/)

+ 88 - 0
src/page/commonfoos.php

@@ -0,0 +1,88 @@
+<?php
+$host = "https://cmd.inp.nsk.su/~compton/gogs";
+$owner = "compton";
+$repo = "tables";
+
+
+function get_request($link)
+{
+    // Common get request with json decoder
+
+    $json = file_get_contents($link);
+    return json_decode($json);
+}
+
+function get_folder_data($host, $owner, $repo, $path, $token)
+{
+    // Gets folder content info
+
+    $obj = get_request("{$host}/api/v1/repos/{$owner}/{$repo}/contents/{$path}?token={$token}");
+    return $obj;
+}
+
+function get_seasons($host, $owner, $repo, $path, $token)
+{
+    // Returns all info about available seasons
+
+    $obj = get_folder_data($host, $owner, $repo, $path, $token);
+    $dir_obj = array_filter($obj, function ($var) {
+        return ($var->type == "dir");
+    });
+    return $dir_obj;
+}
+
+function available_seasons($host, $owner, $repo, $path, $token)
+{
+    // Returns sorted array of available season names
+
+    $season_obj = get_seasons($host, $owner, $repo, $path, $token);
+    foreach ($season_obj as &$s) {
+        $s = $s->name;
+    }
+    usort($season_obj, function ($a, $b) {
+        return substr($a, -4, 4) > substr($b, -4, 4);
+    });
+    return $season_obj;
+}
+
+function season_table_url($host, $owner, $repo, $path, $token, $season)
+{
+    // Returns URL of the season table
+
+    $obj = get_folder_data($host, $owner, $repo, $path, $token);
+    $ret_obj = array_filter($obj, function ($var) use ($season) {
+        return ($var->type == "file") && (substr($var->name, 0, strlen($season)) == $season);
+    });
+
+    $file = current($ret_obj);
+    $download_url_fix = substr($file->download_url, 0, -strlen($file->name)) . $path . "/" . ($file->name);
+    return $download_url_fix;
+}
+
+function energy_point_urls($host, $owner, $repo, $path, $token)
+{
+    // Returns an Array of energy points in $path with "name" keys and "download_url" values
+
+    $obj = get_folder_data($host, $owner, $repo, $path, $token);
+    $obj = array_filter($obj, function ($var) {
+        return ($var->type == "file") && (substr($var->name, -4, 4) == ".csv");
+    });
+
+    $ret_arr = array();
+    foreach ($obj as $file) {
+        $download_url_fix = substr($file->download_url, 0, -strlen($file->name)) . $path . "/" . ($file->name);
+        $ret_arr[$file->name] = $download_url_fix;
+    }
+    // print_r($ret_arr);
+    return $ret_arr;
+}
+
+function isSelected($a, $b)
+{
+    // Utility comparator function for selection lists
+
+    if ($a == $b) {
+        return "selected";
+    }
+    return "";
+}

+ 35 - 0
src/page/index.php

@@ -0,0 +1,35 @@
+<?php
+require __DIR__ . '/commonfoos.php';
+
+$ini_array = parse_ini_file("config.ini");
+$token = $ini_array["apitoken"];
+
+$names = available_seasons($host, $owner, $repo, "energy_points", $token);
+?>
+
+<html>
+<title>Compton pictures</title>
+<meta name="viewport" content="width=device-width, initial-scale=1">
+<link rel="stylesheet" href="main.css">
+
+<body>
+    <div class="main_block">
+        <div class="title_block">
+            <h1>Compton avg by seasons</h1>
+        </div>
+
+        <div class="content_block">
+            <?php
+            foreach ($names as $name) {
+                echo "<div class=\"selection_item\"><a href=\"./init.php?season={$name}\">• {$name}</a></div>";
+            }
+            ?>
+        </div>
+        <br>
+        <?php
+        echo "<div class=\"selection_item\"><a href=\"./summary.php?season=" . end($names) . "\">Season summary</a></div>";
+        ?>
+    </div>
+</body>
+
+</html>

+ 176 - 175
src/page/init.php

@@ -1,171 +1,162 @@
-<!-- https://www.createwithdata.com/chartjs-and-csv/ -->
-<!-- https://towardsdatascience.com/4-ways-to-improve-your-plotly-graphs-517c75947f7e -->
-
 <?php
-$branch = 'dev';
-
-function getsmth($branch){
-    $url = "https://cmd.inp.nsk.su/~compton/gitlist/compton_tables/raw/".$branch."/tables/";
-    $text = file_get_contents($url);
-    $arrays = explode("\n", $text);
-    $clean_arrs = array_filter($arrays, function($value){
-	$temp_arr = explode('/', $value);
-        return preg_match("/[A-Z]+[0-9]+\//", $value);
-    });
-    foreach($clean_arrs as &$val){
-        $val = substr($val, 0, -1);
-    }
-    return $clean_arrs;
-}
-
-$availableSeasons = getsmth($branch);
-
-$season = isset($_GET["season"])&&in_array($_GET["season"], $availableSeasons) ? $_GET["season"] : reset($availableSeasons);
-$url = "https://cmd.inp.nsk.su/~compton/gitlist/compton_tables/raw/".$branch."/tables/".$season."/";
-$url_total_info = "https://cmd.inp.nsk.su/~compton/gitlist/compton_tables/raw/".$branch."/tables/".$season.".csv";
-$text = file_get_contents($url);
-$arrays = explode("\n", $text);
-
-$cleanArrs = array_filter($arrays, function($value) {
-   return end(explode('.', $value)) == "csv";
-});
-
-function isSelected($a, $b){
-    if ($a==$b){
-        return "selected";
-    }
-    return "";
-}
-
-$selected_csv = isset($_GET["csv_file"])&&in_array($_GET["csv_file"], $cleanArrs) ? $_GET["csv_file"] : reset($cleanArrs);
+require __DIR__ . '/commonfoos.php';
+
+$ini_array = parse_ini_file("config.ini");
+$token = $ini_array["apitoken"];
+
+$availableSeasons = available_seasons($host, $owner, $repo, "energy_points", $token);
+
+$season = isset($_GET["season"]) && in_array($_GET["season"], $availableSeasons) ? $_GET["season"] : end($availableSeasons);
+$season_url = season_table_url($host, $owner, $repo, "", $token, $season);
+
+$energyPoints = energy_point_urls($host, $owner, $repo, "energy_points/{$season}", $token);
 
+$selected_csv = isset($_GET["csv_file"]) && array_key_exists($_GET["csv_file"], $energyPoints) ? $_GET["csv_file"] : key($energyPoints);
 ?>
 
 <html>
+
 <head>
     <script src="plotly-latest.min.js"></script>
     <!-- <script src="https://cdn.plot.ly/plotly-latest.min.js"></script> -->
     <meta name="viewport" content="width=device-width, initial-scale=1">
     <link rel="stylesheet" href="main.css">
     <title>Compton interactive plots</title>
-    <!-- <script src="chart.js"></script> -->
 </head>
+
 <body>
     <div id="gd" style="height: max(80vh, 600px); padding-bottom: 20px;"></div>
-            
+
     <form name="form" action="" method="get">
         <p style="text-align: center;">Select energy point:</p>
-	<div style="margin: 0 auto; display: flex; justify-content: center;">
-        <select name="season" class="select-css" style="margin: 0;" onchange="this.form.submit()">
-            <? foreach($availableSeasons as $s){ ?>
-            <option value="<?echo $s?>" <?echo isSelected($s, $season)?>><?echo $s?></option>
-            <? } ?>
-        </select>
-        <select name="csv_file" class="select-css" style="margin: 0;" onchange="this.form.submit()">
-            <? foreach($cleanArrs as $file){ ?>
-            <option value="<?echo $file?>" <?echo isSelected($file, $selected_csv)?>><?echo $file?></option>
-            <? } ?>
-        </select>
-	</div>
+        <div style="margin: 0 auto; display: flex; justify-content: center;">
+            <select name="season" class="select-css" style="margin: 0;" onchange="this.form.submit()">
+                <? foreach ($availableSeasons as $s) { ?>
+                    <option value="<? echo $s ?>" <? echo isSelected($s, $season) ?>>
+                        <? echo $s ?>
+                    </option>
+                <? } ?>
+            </select>
+            <select name="csv_file" class="select-css" style="margin: 0;" onchange="this.form.submit()">
+                <? foreach ($energyPoints as $key => $file) { ?>
+                    <option value="<? echo $key ?>" <? echo isSelected($key, $selected_csv) ?>>
+                        <? echo $key ?>
+                    </option>
+                <? } ?>
+            </select>
+        </div>
     </form>
 
 
     <script>
-        function makeplot(){
-            Plotly.d3.csv('<?echo $url_total_info;?>', (allRows)=>{
-                const {mean_energy, mean_spread, mean_energy_stat_err} = parseResultsTable(allRows, "<?echo $selected_csv;?>");        
-                Plotly.d3.csv('<?echo $url.$selected_csv;?>', function(data){processData(data, mean_energy, mean_spread, mean_energy_stat_err)});
-            }
-            );
+        function makeplot() {
+            Plotly.d3.csv('<? echo $season_url; ?>', (allRows) => {
+                const {
+                    mean_energy,
+                    mean_spread,
+                    mean_energy_stat_err
+                } = parseResultsTable(allRows, "<? echo $selected_csv; ?>");
+                Plotly.d3.csv('<? echo $energyPoints[$selected_csv]; ?>', function(data) {
+                    processData(data, mean_energy, mean_spread, mean_energy_stat_err)
+                });
+            });
         }
-        
-        function parseResultsTable(allRows, energy_point){
+
+        function parseResultsTable(allRows, energy_point) {
             // Extracts a row following the energy point in the total csv file (or null if energy point is not exists)
             data = energy_point.slice(0, -4).split('_');
-	    for (var i=0; i<allRows.length; i++){
-                if (allRows[i].first_run == data[1] ){
+            for (var i = 0; i < allRows.length; i++) {
+                if (allRows[i].first_run == data[1]) {
                     return allRows[i];
                 }
             }
             return null;
         }
-        
-        function parseRow(row){
+
+        function parseRow(row) {
             // Parses a row from the detailed csv file
-	    row['start_time'] = Date.parse(row['compton_start']);
+            row['start_time'] = Date.parse(row['compton_start']);
             row['stop_time'] = Date.parse(row['compton_stop']);
-            row['center_time'] = new Date((row['start_time'] + row['stop_time'])/2);
-            row['timedelta'] = (row['stop_time'] - row['start_time'])/2/1000; // in seconds
+            row['center_time'] = new Date((row['start_time'] + row['stop_time']) / 2);
+            row['timedelta'] = (row['stop_time'] - row['start_time']) / 2 / 1000; // in seconds
 
-	    text_energy = "E = " + parseFloat(row['e_mean']).toFixed(2) + "±" + parseFloat(row['e_std']).toFixed(2) + " MeV<br>";
-	    text_spread = "spread = " + parseFloat(row['spread_mean']).toFixed(2) + "±" + parseFloat(row['spread_std']).toFixed(2) + " MeV<br>";
-            text_str = text_energy + text_spread + "<b>Compton</b><br>" + "<i>Start: </i>" + 
-                    row['compton_start'] + "<br><i>Stop: </i>" + row['compton_stop'] + "<br><br>";
+            text_energy = "E = " + parseFloat(row['e_mean']).toFixed(2) + "±" + parseFloat(row['e_std']).toFixed(2) + " MeV<br>";
+            text_spread = "spread = " + parseFloat(row['spread_mean']).toFixed(2) + "±" + parseFloat(row['spread_std']).toFixed(2) + " MeV<br>";
+            text_str = text_energy + text_spread + "<b>Compton</b><br>" + "<i>Start: </i>" +
+                row['compton_start'] + "<br><i>Stop: </i>" + row['compton_stop'] + "<br><br>";
             row['text_str'] = text_str + "<b>Runs: </b>" + row['run_first'] + " - " + row['run_last'] + "<br><br>";
             return row;
         }
-        
-        function processSpread(data, elementId, mean_value){
-            let x = [], y = [], std_y = []
-            for (var i=0; i<data.length; i++){
-                const {center_time, spread_mean, spread_std} = parseRow(data[i]);
+
+        function processSpread(data, elementId, mean_value) {
+            let x = [],
+                y = [],
+                std_y = []
+            for (var i = 0; i < data.length; i++) {
+                const {
+                    center_time,
+                    spread_mean,
+                    spread_std
+                } = parseRow(data[i]);
                 x.push(center_time);
                 y.push(spread_mean);
                 std_y.push(spread_std);
             }
             makeSpreadPlot(elementId, x, y, std_y, mean_value);
         }
-        
+
         function kde(x, y, w) {
-            const ts = (t) => t.getTime()/1000;
+            const ts = (t) => t.getTime() / 1000;
             const toDateTime = (secs) => {
                 let t = new Date(0);
                 t.setSeconds(secs);
                 return t;
             };
             const steps = 1000;
-            const dt =  (ts(x[x.length - 1]) - ts(x[0]))/steps;
+            const dt = (ts(x[x.length - 1]) - ts(x[0])) / steps;
             const kernel = (x, x0, w0, y0) => {
-                if (Math.abs(x-x0)>w0){
+                if (Math.abs(x - x0) > w0) {
                     return 0;
                 }
-                return y0/w0;
+                return y0 / w0;
                 //return y0*3*(1-((x-x0)/w0/2)**2)/4;
-            }; 
+            };
             const get_est = (timestamp) => {
                 let val = 0
-                for (var i=0; i<x.length; i++){
+                for (var i = 0; i < x.length; i++) {
                     val += kernel(timestamp, ts(x[i]), w[i], y[i]);
                 }
                 return val;
             };
             //console.log(x, y);
-            const timestamp_arr = Plotly.d3.range(steps).map(function(i){return ts(x[0])+i*dt;});
-            
+            const timestamp_arr = Plotly.d3.range(steps).map(function(i) {
+                return ts(x[0]) + i * dt;
+            });
+
             let kdex = [];
             let kdey = [];
-            for (var j=0; j<timestamp_arr.length; j++){
+            for (var j = 0; j < timestamp_arr.length; j++) {
                 kdex.push(toDateTime(timestamp_arr[j]));
                 kdey.push(get_est(timestamp_arr[j]));
             }
             //console.log(kdex, kdey);
             return [kdex, kdey]
         }
-        
-        function oldAverage(E, L){
+
+        function oldAverage(E, L) {
             //Averager by the old method with E and L only
-            if (E.length !== L.length){
+            if (E.length !== L.length) {
                 return null;
             }
             let EL = 0;
             let sL = 0;
-            for (let i = 0; i<E.length; i++){
-                EL += parseFloat(E[i])*parseFloat(L[i]);
+            for (let i = 0; i < E.length; i++) {
+                EL += parseFloat(E[i]) * parseFloat(L[i]);
                 sL += parseFloat(L[i]);
             }
-            return EL/sL;
+            return EL / sL;
         }
-        
+
         function processData(allRows, mean_energy, mean_spread, mean_energy_stat_err) {
             // Processes all data rows
             var dict = {};
@@ -177,44 +168,44 @@ $selected_csv = isset($_GET["csv_file"])&&in_array($_GET["csv_file"], $cleanArrs
             dict['compton'] = [];
             dict['lum'] = [];
             dict['twidth'] = [];
-            
-            for (var i=0; i<allRows.length; i++){
+
+            for (var i = 0; i < allRows.length; i++) {
                 const row = parseRow(allRows[i]);
-                                
-                dict['x'].push( row['center_time'] );
-                dict['e_mean'].push( row['e_mean'] );
-                dict['e_std'].push( row['e_std'] );
-                dict['spread_mean'].push( row['spread_mean'] );
-                dict['spread_std'].push( row['spread_std'] );
-                dict['compton'].push( row['text_str'] );
-                dict['lum'].push( row['luminosity'] );
-                dict['twidth'].push( row['timedelta'] );
+
+                dict['x'].push(row['center_time']);
+                dict['e_mean'].push(row['e_mean']);
+                dict['e_std'].push(row['e_std']);
+                dict['spread_mean'].push(row['spread_mean']);
+                dict['spread_std'].push(row['spread_std']);
+                dict['compton'].push(row['text_str']);
+                dict['lum'].push(row['luminosity']);
+                dict['twidth'].push(row['timedelta']);
             }
-            
+
             const [a, b] = kde(dict['x'], dict['lum'], dict['twidth']);
             dict['kdex'] = a;
             dict['kdey'] = b;
             //console.log(dict['kdex'], dict['kdey']);
             //oldAverage(y, dict['lum']);
-            
+
             dict['mean_energy_total'] = mean_energy;
             dict['old_mean_energy_total'] = oldAverage(dict['e_mean'], dict['lum']);
             dict['mean_spread_total'] = mean_spread;
-	    dict['mean_energy_stat_err'] = mean_energy_stat_err;
-            
+            dict['mean_energy_stat_err'] = mean_energy_stat_err;
+
             makePlotly(dict, "gd");
         }
-        
-        function makePlotly(dict, elementId){
+
+        function makePlotly(dict, elementId) {
             const getYRange = (y, std_y) => {
                 const ys = [...y].sort();
                 const std_ys = [...std_y].sort();
-                let idx = Math.floor(ys.length/2);
+                let idx = Math.floor(ys.length / 2);
                 const y0 = parseFloat(ys[idx]);
                 const std0 = parseFloat(std_ys[idx]);
-                return [y0-6*std0, y0+6*std0];
+                return [y0 - 6 * std0, y0 + 6 * std0];
             };
-            
+
             var trace1 = {
                 x: dict['x'],
                 y: dict['e_mean'],
@@ -268,32 +259,38 @@ $selected_csv = isset($_GET["csv_file"])&&in_array($_GET["csv_file"], $cleanArrs
                 type: "scatter",
             };
             var traces = [trace1, trace2, trace3];
-            
+
             var updatemenus = [];
-	    console.log(dict);
-            if (dict['mean_energy_total']){
+            console.log(dict);
+            if (dict['mean_energy_total']) {
                 updatemenus = [{
-                buttons: [
-                    {
-                        args:[{'shapes[0].visible': true, 'shapes[1].visible': false, 'title': '<E> = ' + parseFloat(dict['mean_energy_total']).toFixed(3) + '±' + parseFloat(dict['mean_energy_stat_err']).toFixed(3) + ' MeV, ' + '<Spread> = ' + parseFloat(dict['mean_spread_total']).toFixed(3) + ' MeV',}],
+                    buttons: [{
+                        args: [{
+                            'shapes[0].visible': true,
+                            'shapes[1].visible': false,
+                            'title': '<E> = ' + parseFloat(dict['mean_energy_total']).toFixed(3) + '±' + parseFloat(dict['mean_energy_stat_err']).toFixed(3) + ' MeV, ' + '<Spread> = ' + parseFloat(dict['mean_spread_total']).toFixed(3) + ' MeV',
+                        }],
                         label: 'Current average method',
                         method: 'relayout'
                     }, {
-                        args:[{'shapes[0].visible': false, 'shapes[1].visible': true, 'title': '<E> = ' + parseFloat(dict['old_mean_energy_total']).toFixed(3) + ' MeV, ' + '<Spread> = ' + parseFloat(dict['mean_spread_total']).toFixed(3) + ' MeV',}],
+                        args: [{
+                            'shapes[0].visible': false,
+                            'shapes[1].visible': true,
+                            'title': '<E> = ' + parseFloat(dict['old_mean_energy_total']).toFixed(3) + ' MeV, ' + '<Spread> = ' + parseFloat(dict['mean_spread_total']).toFixed(3) + ' MeV',
+                        }],
                         label: 'Former average method',
                         method: 'relayout'
-                    },
-                ],
-                direction: 'center',
-                showactive: 'true',
-                type: 'dropdown',
-                y: 1.1,
-                xanchor: 'left',
-                yanchor: 'top',
-                active: 0,
-            }];
+                    }, ],
+                    direction: 'center',
+                    showactive: 'true',
+                    type: 'dropdown',
+                    y: 1.1,
+                    xanchor: 'left',
+                    yanchor: 'top',
+                    active: 0,
+                }];
             }
-            
+
             var layout = {
                 title: '<E> = ' + parseFloat(dict['mean_energy_total']).toFixed(3) + '±' + parseFloat(dict['mean_energy_stat_err']).toFixed(3) + ' MeV, ' + '<Spread> = ' + parseFloat(dict['mean_spread_total']).toFixed(3) + ' MeV',
                 updatemenus: updatemenus,
@@ -333,55 +330,59 @@ $selected_csv = isset($_GET["csv_file"])&&in_array($_GET["csv_file"], $cleanArrs
                 plot_bgcolor: 'rgba(0,0,0,0)',
                 autosize: true,
             };
-            
-            if (dict['mean_energy_total']){
+
+            if (dict['mean_energy_total']) {
                 layout['shapes'] = [{
-                    type: 'line',
-                    yref: 'y3',
-                    xref: 'paper',
-                    x0: 0,
-                    x1: 1,
-                    y0: dict['mean_energy_total'],
-                    y1: dict['mean_energy_total'],
-                    line: {
-                        color: '#590A0A',
+                        type: 'line',
+                        yref: 'y3',
+                        xref: 'paper',
+                        x0: 0,
+                        x1: 1,
+                        y0: dict['mean_energy_total'],
+                        y1: dict['mean_energy_total'],
+                        line: {
+                            color: '#590A0A',
+                        },
                     },
-                },
-                {
-                    type: 'line',
-                    yref: 'y3',
-                    xref: 'paper',
-                    x0: 0,
-                    x1: 1,
-                    y0: dict['old_mean_energy_total'],
-                    y1: dict['old_mean_energy_total'],
-                    line: {
-                        color: '#590A0A',
-                    },
-                    visible: false,
-                },
-                {
-                    type: 'line',
-                    yref: 'y2',
-                    xref: 'paper',
-                    x0: 0,
-                    x1: 1,
-                    y0: dict['mean_spread_total'],
-                    y1: dict['mean_spread_total'],
-                    line: {
-                        color: '#590A0A',
+                    {
+                        type: 'line',
+                        yref: 'y3',
+                        xref: 'paper',
+                        x0: 0,
+                        x1: 1,
+                        y0: dict['old_mean_energy_total'],
+                        y1: dict['old_mean_energy_total'],
+                        line: {
+                            color: '#590A0A',
+                        },
+                        visible: false,
                     },
-                    visible: true,
-                }];
+                    {
+                        type: 'line',
+                        yref: 'y2',
+                        xref: 'paper',
+                        x0: 0,
+                        x1: 1,
+                        y0: dict['mean_spread_total'],
+                        y1: dict['mean_spread_total'],
+                        line: {
+                            color: '#590A0A',
+                        },
+                        visible: true,
+                    }
+                ];
             }
-            
-            Plotly.newPlot('gd', traces, layout, {modeBarButtonsToRemove: ['toImage'], responsive: true,});
-            
+
+            Plotly.newPlot('gd', traces, layout, {
+                modeBarButtonsToRemove: ['toImage'],
+                responsive: true,
+            });
+
         }
-        
+
         makeplot();
-        
     </script>
 
 </body>
-</html>
+
+</html>

+ 73 - 51
src/page/main.css

@@ -1,102 +1,124 @@
 @import url('https://fonts.googleapis.com/css2?family=Raleway&display=swap');
 @import url('https://fonts.googleapis.com/css2?family=Open+Sans&display=swap');
 
-a{
-    font-size: 36px;
+a {
+    font-size: 3.6rem;
     text-decoration: none;
     color: #26110C;
-    transition-duration: 1s;
     font-family: 'Raleway', sans-serif;
 }
 
-a:hover{
+a:hover {
     color: #732727;
+    text-decoration: underline;
 }
 
-a:visited{
-    color: #26110C;
+body {
+    background-color: #F2A76650;
+    margin: 0;
+    /*#D9B9A7;*/
 }
 
-body{
-    background-color: #F2A76687; /*#D9B9A7;*/
+br {
+    margin-bottom: 1.8rem;
 }
 
-br{
-    margin-bottom: 18px;
+div.content_block {
+    display: flex;
+    flex-direction: column-reverse;
+    flex-wrap: wrap;
+    flex-basis: 100%;
+    flex: 1;
+    width: 100%;
 }
 
-div.main_block{
-    width: max(50%, 500px);
+div.main_block {
+    width: max(50%, 350px);
     max-width: min(100%, 800px);
     margin: 0 auto;
-    padding-top: 50px;
 }
 
-h1{
+div.selection_item {
+    min-width: 40%;
+    padding: 0.5rem 1rem;
+}
+
+div.title_block {
+    margin: 4rem 0;
+}
+
+h1 {
     font-family: 'Raleway', sans-serif;
-    font-size: 48px;
+    font-size: 4.8rem;
+    padding-left: 1rem;
+    margin: 0;
 }
 
-h2{
+h2 {
     font-family: 'Raleway', sans-serif;
-    font-size: 40px;
+    font-size: 4rem;
 }
 
-h3{
+h3 {
     font-family: 'Open Sans', sans-serif;
-    font-size: 32px;  
+    font-size: 3.2rem;
 }
 
-h4{
+h4 {
     font-family: 'Open Sans', sans-serif;
-    font-size: 24px;
+    font-size: 2.4rem;
 }
 
-hr{
+hr {
     border: 0;
     border-bottom: dashed;
 }
 
-img{
-    padding: 15px 0;
+html {
+    font-size: 62.5%;
+    margin: 0;
+}
+
+img {
+    padding: 1.5rem 0;
     width: 100%;
 }
 
-p{
+p {
     font-family: 'Raleway', sans-serif;
-    font-size: 28px;
+    font-size: 2.8rem;
 }
 
 select {
 
-  /* styling */
-  background-color: #F2A76652;
-  border: thin solid #F23030;
-  border-radius: 4px;
-  display: block;
-  font: inherit;
-  font-size: 20px;
-  line-height: 1.5em;
-  padding: 0.5em 3.5em 0.5em 1em;
+    /* styling */
+    background-color: #F2A76652;
+    border: thin solid #F23030;
+    border-radius: 4px;
+    display: block;
+    font: inherit;
+    font-size: 2.0rem;
+    line-height: 1.5em;
+    padding: 0.5em 3.5em 0.5em 1em;
 
-  /* reset */
+    /* reset */
 
-  margin: 0 auto;      
-  -webkit-box-sizing: border-box;
-  -moz-box-sizing: border-box;
-  box-sizing: border-box;
-  -webkit-appearance: none;
-  -moz-appearance: none;
+    margin: 0 auto;
+    -webkit-box-sizing: border-box;
+    -moz-box-sizing: border-box;
+    box-sizing: border-box;
+    -webkit-appearance: none;
+    -moz-appearance: none;
 }
 
 select.select-css {
     background-image:
-	linear-gradient(45deg, transparent 50%, #260101 0%),
-	linear-gradient(135deg, #260101 50%, transparent 0%);
-  background-position:
-      calc(100% - 21px) calc(1em + 2px),
-      calc(100% - 13px) calc(1em + 2px);
-  background-size:
-      8px 8px, 8px 8px, 2.5em 2.5em;
-  background-repeat: no-repeat;
-}
+        linear-gradient(45deg, transparent 50%, #260101 0%),
+        linear-gradient(135deg, #260101 50%, transparent 0%);
+    background-position:
+        calc(100% - 21px) calc(1em + 2px),
+        calc(100% - 13px) calc(1em + 2px);
+    background-size:
+        8px 8px, 8px 8px, 2.5em 2.5em;
+    background-repeat: no-repeat;
+}

+ 0 - 350
src/page/plots.php

@@ -1,350 +0,0 @@
-<!-- https://www.createwithdata.com/chartjs-and-csv/ -->
-<!-- https://towardsdatascience.com/4-ways-to-improve-your-plotly-graphs-517c75947f7e -->
-
-<?php
-$selected_csv = 
-
-$branch = 'dev';
-$season = 'NNBAR2021';
-$url = "https://cmd.inp.nsk.su/~compton/gitlist/compton_tables/raw/".$branch."/tables/".$season."/";
-$url_total_info = "https://cmd.inp.nsk.su/~compton/gitlist/compton_tables/raw/".$branch."/tables/".$season.".csv";
-$text = file_get_contents($url);
-$arrays = explode("\n", $text);
-
-$cleanArrs = array_filter($arrays, function($value) {
-   return end(explode('.', $value)) == "csv";
-});
-
-function isSelected($a, $b){
-    if ($a==$b){
-        return "selected";
-    }
-    return "";
-}
-
-$selected_csv = isset($_GET["csv_file"]) ? $_GET["csv_file"] : reset($cleanArrs);
-
-?>
-
-<html>
-<head>
-    <!--<script src="plotly-2.11.1.min.js"></script> -->
-    <script src="https://cdn.plot.ly/plotly-latest.min.js"></script>
-    <meta name="viewport" content="width=device-width, initial-scale=1">
-    <link rel="stylesheet" href="main.css">
-    <title>Compton interactive plots</title>
-    <!-- <script src="chart.js"></script> -->
-</head>
-<body>
-    <div id="gd" style="height: max(80vh, 600px); padding-bottom: 20px;"></div>
-            
-    <form name="form" action="" method="get">
-        <p style="text-align: center;">Select energy point:</p>
-        <select name="csv_file" class="select-css" onchange="this.form.submit()">
-            <?
-            foreach($cleanArrs as $file){
-                ?><option value="<?echo $file?>" <?echo isSelected($file, $selected_csv)?>><?echo $file?></option>
-                <?
-            }
-            ?>
-        </select>
-    </form>
-
-
-    <script>
-        function makeplot(){
-            Plotly.d3.csv('<?echo $url_total_info;?>', (allRows)=>{
-                const {mean_energy, mean_spread} = parseResultsTable(allRows, <?echo reset(explode('_', $selected_csv));?>);        
-                Plotly.d3.csv('<?echo $url.$selected_csv;?>', function(data){processData(data, mean_energy, mean_spread)});
-            }
-            );
-        }
-        
-        function parseResultsTable(allRows, energy_point){
-            // Extracts a row following the energy point in the total csv file (or null if energy point is not exists)
-            for (var i=0; i<allRows.length; i++){
-                if (allRows[i].energy_point == energy_point ){
-                    return allRows[i];
-                }
-            }
-            return null;
-        }
-        
-        function parseRow(row){
-            // Parses a row from the detailed csv file
-            row['start_time'] = Date.parse(row['compton_start']);
-            row['stop_time'] = Date.parse(row['compton_stop']);
-            row['center_time'] = new Date((row['start_time'] + row['stop_time'])/2);
-            row['timedelta'] = (row['stop_time'] - row['start_time'])/2/1000; // in seconds
-            text_str = "<b>Compton</b><br>" + "<i>Start: </i>" + 
-                    row['compton_start'] + "<br><i>Stop: </i>" + row['compton_stop'] + "<br><br>";
-            row['text_str'] = text_str + "<b>Runs: </b>" + row['run_first'] + " - " + row['run_last'] + "<br><br>";
-            return row;
-        }
-        
-        function processSpread(data, elementId, mean_value){
-            let x = [], y = [], std_y = []
-            for (var i=0; i<data.length; i++){
-                const {center_time, spread_mean, spread_std} = parseRow(data[i]);
-                x.push(center_time);
-                y.push(spread_mean);
-                std_y.push(spread_std);
-            }
-            makeSpreadPlot(elementId, x, y, std_y, mean_value);
-        }
-        
-        function kde(x, y, w) {
-            const ts = (t) => t.getTime()/1000;
-            const toDateTime = (secs) => {
-                let t = new Date(0);
-                t.setSeconds(secs);
-                return t;
-            };
-            const steps = 1000;
-            const dt =  (ts(x[x.length - 1]) - ts(x[0]))/steps;
-            const kernel = (x, x0, w0, y0) => {
-                if (Math.abs(x-x0)>w0){
-                    return 0;
-                }
-                return y0/w0;
-                //return y0*3*(1-((x-x0)/w0/2)**2)/4;
-            }; 
-            const get_est = (timestamp) => {
-                let val = 0
-                for (var i=0; i<x.length; i++){
-                    val += kernel(timestamp, ts(x[i]), w[i], y[i]);
-                }
-                return val;
-            };
-            //console.log(x, y);
-            const timestamp_arr = Plotly.d3.range(steps).map(function(i){return ts(x[0])+i*dt;});
-            
-            let kdex = [];
-            let kdey = [];
-            for (var j=0; j<timestamp_arr.length; j++){
-                kdex.push(toDateTime(timestamp_arr[j]));
-                kdey.push(get_est(timestamp_arr[j]));
-            }
-            //console.log(kdex, kdey);
-            return [kdex, kdey]
-        }
-        
-        function oldAverage(E, L){
-            //Averager by the old method with E and L only
-            if (E.length !== L.length){
-                return null;
-            }
-            let EL = 0;
-            let sL = 0;
-            for (let i = 0; i<E.length; i++){
-                EL += parseFloat(E[i])*parseFloat(L[i]);
-                sL += parseFloat(L[i]);
-            }
-            return EL/sL;
-        }
-        
-        function processData(allRows, mean_energy, mean_spread) {
-            // Processes all data rows
-            var dict = {};
-            dict['x'] = [];
-            dict['e_mean'] = [];
-            dict['e_std'] = [];
-            dict['spread_mean'] = [];
-            dict['spread_std'] = [];
-            dict['compton'] = [];
-            dict['lum'] = [];
-            dict['twidth'] = [];
-            
-            for (var i=0; i<allRows.length; i++){
-                const row = parseRow(allRows[i]);
-                                
-                dict['x'].push( row['center_time'] );
-                dict['e_mean'].push( row['e_mean'] );
-                dict['e_std'].push( row['e_std'] );
-                dict['spread_mean'].push( row['spread_mean'] );
-                dict['spread_std'].push( row['spread_std'] );
-                dict['compton'].push( row['text_str'] );
-                dict['lum'].push( row['luminosity'] );
-                dict['twidth'].push( row['timedelta'] );
-            }
-            
-            const [a, b] = kde(dict['x'], dict['lum'], dict['twidth']);
-            dict['kdex'] = a;
-            dict['kdey'] = b;
-            //console.log(dict['kdex'], dict['kdey']);
-            //oldAverage(y, dict['lum']);
-            
-            dict['mean_energy_total'] = mean_energy;
-            dict['old_mean_energy_total'] = oldAverage(dict['e_mean'], dict['lum']);
-            dict['mean_spread_total'] = mean_spread;
-            
-            makePlotly(dict, "gd");
-        }
-        
-        function makePlotly(dict, elementId){
-            const getYRange = (y, std_y) => {
-                const ys = [...y].sort();
-                const std_ys = [...std_y].sort();
-                let idx = Math.floor(ys.length/2);
-                const y0 = parseFloat(ys[idx]);
-                const std0 = parseFloat(std_ys[idx]);
-                return [y0-6*std0, y0+6*std0];
-            };
-            
-            var trace1 = {
-                x: dict['x'],
-                y: dict['e_mean'],
-                yaxis: 'y3',
-                mode: 'markers',
-                text: dict['compton'],
-                hovertemplate: "%{text}<br><br>" + "<extra></extra>",
-                hovermode: "x",
-                error_y: {
-                    type: 'data',
-                    array: dict['e_std'],
-                    color: '#260101',
-                },
-                showlegend: false,
-                marker: {
-                    color: '#260101',
-                },
-                type: "scatter",
-            };
-            var trace2 = {
-                x: dict['x'],
-                y: dict['spread_mean'],
-                yaxis: 'y2',
-                mode: 'markers',
-                text: dict['compton'],
-                hovertemplate: "%{text}<br><br>" + "<extra></extra>",
-                hovermode: "x",
-                error_y: {
-                    type: 'data',
-                    array: dict['spread_std'],
-                    color: '#260101',
-                },
-                showlegend: false,
-                marker: {
-                    color: '#260101',
-                },
-                type: "scatter",
-            };
-            var trace3 = {
-                x: dict['kdex'],
-                y: dict['kdey'],
-                hovertemplate: "%{y}<br><br>" + "<extra></extra>",
-                hovermode: "x",
-                showlegend: false,
-                marker: {
-                    color: '#F23030',
-                },
-                line: {
-                    shape: 'hvh',
-                },
-                type: "scatter",
-            };
-            var traces = [trace1, trace2, trace3];
-            
-            var updatemenus = [];
-            if (dict['mean_energy_total']){
-                updatemenus = [{
-                buttons: [
-                    {
-                        args:[{'shapes[0].visible': true, 'shapes[1].visible': false, 'title': 'Mean energy: ' + parseFloat(dict['mean_energy_total']).toFixed(3) + ' MeV',}],
-                        label: 'Current average method',
-                        method: 'relayout'
-                    }, {
-                        args:[{'shapes[0].visible': false, 'shapes[1].visible': true, 'title': 'Mean energy: ' + dict['old_mean_energy_total'].toFixed(3)  + ' MeV',}],
-                        label: 'Former average method',
-                        method: 'relayout'
-                    },
-                ],
-                direction: 'center',
-                showactive: 'true',
-                type: 'dropdown',
-                y: 1.1,
-                xanchor: 'left',
-                yanchor: 'top',
-                active: 0,
-            }];
-            }
-            
-            
-            var layout = {
-                title: 'Mean energy: ' + dict['mean_energy_total'] + ' MeV',
-                updatemenus: updatemenus,
-                font: {
-                    size: 18,
-                },
-                xaxis: {
-                    title: "Time, NSK",
-                    automargin: true,
-                },
-                yaxis3: {
-                    domain: [0.6, 1],
-                    title: "Mean energy, MeV",
-                    automargin: true,
-                    //showspikes: true,
-                    //spikemode: "across",
-                    //spikesnap: "data",
-                },
-                yaxis2: {
-                    domain: [0.3, 0.5],
-                    title: "Spread, MeV",
-                    autorange: false,
-                    range: getYRange(dict['spread_mean'], dict['spread_std']),
-                    //showspikes: true,
-                    //spikemode: "across",
-                    //spikesnap: "data",
-                },
-                yaxis: {
-                    domain: [0, 0.2],
-                    automargin: true,
-                    zeroline: true,
-                    rangemode: 'positive',
-                    title: "L, nb<sup>-1</sup>/s",
-                    hoverformat: '.2f',
-                },
-                paper_bgcolor: 'rgba(0,0,0,0)',
-                plot_bgcolor: 'rgba(0,0,0,0)',
-                autosize: true,
-            };
-            
-            if (dict['mean_energy_total']){
-                layout['shapes'] = [{
-                    type: 'line',
-                    yref: 'y3',
-                    xref: 'paper',
-                    x0: 0,
-                    x1: 1,
-                    y0: dict['mean_energy_total'],
-                    y1: dict['mean_energy_total'],
-                    line: {
-                        color: '#590A0A',
-                    },
-                },
-                {
-                    type: 'line',
-                    yref: 'y3',
-                    xref: 'paper',
-                    x0: 0,
-                    x1: 1,
-                    y0: dict['old_mean_energy_total'],
-                    y1: dict['old_mean_energy_total'],
-                    line: {
-                        color: '#590A0A',
-                    },
-                    visible: false,
-                }];
-            }
-            
-            Plotly.newPlot('gd', traces, layout, {modeBarButtonsToRemove: ['toImage'], responsive: true,});
-            
-        }
-        
-        makeplot();
-        
-    </script>
-
-</body>
-</html>

+ 127 - 117
src/page/summary.php

@@ -1,30 +1,18 @@
 <?php
-$selected_csv = 
+require __DIR__ . '/commonfoos.php';
 
-$branch = 'dev';
-$season = 'NNBAR2021';
-$url = "https://cmd.inp.nsk.su/~compton/gitlist/compton_tables/raw/".$branch."/tables/";
-$url_total_info = "https://cmd.inp.nsk.su/~compton/gitlist/compton_tables/raw/".$branch."/tables/".$season.".csv";
-$text = file_get_contents($url);
-$arrays = explode("\n", $text);
+$ini_array = parse_ini_file("config.ini");
+$token = $ini_array["apitoken"];
 
-$cleanArrs = array_filter($arrays, function($value) {
-   return end(explode('.', $value)) == "csv";
-});
+$available_seasons = available_seasons($host, $owner, $repo, "energy_points", $token);
 
-function isSelected($a, $b){
-    if ($a==$b){
-        return "selected";
-    }
-    return "";
-}
-
-$selected_csv = isset($_GET["csv_file"]) ? $_GET["csv_file"] : reset($cleanArrs);
-$selected_season = substr($selected_csv, 0, -4);
+$selected_season = isset($_GET["season"]) && in_array($_GET["season"], $available_seasons) ? $_GET["season"] : end($available_seasons);
+$selected_season_url = season_table_url($host, $owner, $repo, "", $token, $selected_season);
 
 ?>
 
 <html>
+
 <head>
     <script src="plotly-latest.min.js"></script>
     <script src="fmin.min.js"></script>
@@ -32,105 +20,113 @@ $selected_season = substr($selected_csv, 0, -4);
     <meta name="viewport" content="width=device-width, initial-scale=1">
     <link rel="stylesheet" href="main.css">
     <title>Compton interactive plots</title>
-    <!-- <script src="chart.js"></script> -->
 </head>
+
 <body>
 
-<div style="display: flex; align-items: center; flex-wrap: wrap; justify-content: center; margin-top: min(20px,2%); margin-bottom: 40px;">
-    <h2 style="margin: 0;"><a href="index.php" style="font-size: inherit;">Pictures</a> / Season</h2>
-    <form name="form" action="" style="padding: 0 15px; margin: auto 0;" method="get">
-        <select name="csv_file" class="select-css" style="padding: 0.3em 1.3em 0.3em 0.4em;" onchange="this.form.submit()">
-            <?
-            foreach($cleanArrs as $file){
-                ?><option value="<?echo $file?>" <?echo isSelected($file, $selected_csv)?>><?echo substr($file, 0, -4)?></option>
+    <div style="display: flex; align-items: center; flex-wrap: wrap; justify-content: center; margin-top: min(20px,2%); margin-bottom: 40px;">
+        <h2 style="margin: 0;"><a href="./" style="font-size: inherit;">Pictures</a> / Season</h2>
+        <form name="form" action="" style="padding: 0 15px; margin: auto 0;" method="get">
+            <select name="season" class="select-css" style="padding: 0.3em 1.3em 0.3em 0.4em;" onchange="this.form.submit()">
                 <?
-            }
-            ?>
-        </select>
-    </form>
-    <h2 style="margin: 0;">summary</h2>
-</div>
-
-<div style="display: flex; align-items: center; flex-wrap: wrap; justify-content: center;">
-    <div style="max-width: 900px; width: 45%; min-width: min(100%, 600px); margin: 0 max(1%, 10px);">
-        <h3>Mean energy measurements</h3>
-        <div id="mean_energy_plot"></div>
-    </div>
-    <div style="max-width: 900px; width: 45%; min-width: min(100%, 600px); margin: 0 max(1%, 10px);">
-        <h3>Spread measurements</h3>
-        <div id="spread_plot"></div>
-    </div>
-    <div style="max-width: 900px; width: 45%; min-width: min(100%, 600px); margin: 0 max(1%, 10px);">
-        <h3>Mean energy deviations</h3>
-        <div id="energy_dev_plot"></div>
+                foreach ($available_seasons as $season) {
+                ?><option value="<? echo $season ?>" <? echo isSelected($season, $selected_season) ?>><? echo $season ?></option>
+                <?
+                }
+                ?>
+            </select>
+        </form>
+        <h2 style="margin: 0;">summary</h2>
     </div>
-    <div style="max-width: 900px; width: 45%; min-width: min(100%, 600px); margin: 0 max(1%, 10px);">
-        <h3>Impact of mean energy deviations</h3>
-        <div id="impact_energy_dev_plot"></div>
+
+    <div style="display: flex; align-items: center; flex-wrap: wrap; justify-content: center;">
+        <div style="max-width: 900px; width: 45%; min-width: min(100%, 600px); margin: 0 max(1%, 10px);">
+            <h3>Mean energy measurements</h3>
+            <div id="mean_energy_plot"></div>
+        </div>
+        <div style="max-width: 900px; width: 45%; min-width: min(100%, 600px); margin: 0 max(1%, 10px);">
+            <h3>Spread measurements</h3>
+            <div id="spread_plot"></div>
+        </div>
+        <div style="max-width: 900px; width: 45%; min-width: min(100%, 600px); margin: 0 max(1%, 10px);">
+            <h3>Mean energy deviations</h3>
+            <div id="energy_dev_plot"></div>
+        </div>
+        <div style="max-width: 900px; width: 45%; min-width: min(100%, 600px); margin: 0 max(1%, 10px);">
+            <h3>Impact of mean energy deviations</h3>
+            <div id="impact_energy_dev_plot"></div>
+        </div>
     </div>
-</div>
-<div style="margin-top: 40px; width: 100%;">
-    <div style="width: min(100%, 600px); margin: 0 auto;">
-        <h3 style="padding-right: 40px;">Methodology</h3>
-        <p style="font-size: inherit;"><b>Mean energy measurements:</b> a difference between measured mean beam energy and target beam energy vs target beam energy.</p>
-        <p style="font-size: inherit;"><b>Spread measurements:</b> a measured mean beam spread vs measured mean beam energy for the given energy points. A line shows the best fit of these points to a linear function (as a result of chi-square minimization)</p>
-        <p style="font-size: inherit;"><b>Mean energy deviations:</b> a measured mean energy deviations (so-called mean_energy_sys_err) vs measured mean beam energy for the given energy points.</p>
-        <p style="font-size: inherit;"><b>Impact of mean energy deviations:</b> a part of the mean energy deviation in total uncertainty (as d/√(d<sup>2</sup> + s<sup>2</sup>), where d is the mean energy deviation and s is the mean spread) for the given energy points.</p>
+    <div style="margin-top: 40px; width: 100%;">
+        <div style="width: min(100%, 600px); margin: 0 auto; font-size: 2rem;">
+            <h3 style="padding-right: 40px;">Methodology</h3>
+            <p style="font-size: inherit;"><b>Mean energy measurements:</b> a difference between measured mean beam energy and target beam energy vs target beam energy.</p>
+            <p style="font-size: inherit;"><b>Spread measurements:</b> a measured mean beam spread vs measured mean beam energy for the given energy points. A line shows the best fit of these points to a linear function (as a result of chi-square minimization)</p>
+            <p style="font-size: inherit;"><b>Mean energy deviations:</b> a measured mean energy deviations (so-called mean_energy_sys_err) vs measured mean beam energy for the given energy points.</p>
+            <p style="font-size: inherit;"><b>Impact of mean energy deviations:</b> a part of the mean energy deviation in total uncertainty (as d/√(d<sup>2</sup> + s<sup>2</sup>), where d is the mean energy deviation and s is the mean spread) for the given energy points.</p>
+        </div>
     </div>
-</div>
 
-<script>
-        function makeplot(){
-            Plotly.d3.csv('<?echo $url.$selected_csv;?>', (allRows)=>{
-                const {point_name, energy_point, mean_energy, mean_energy_stat_err, mean_energy_sys_err, mean_spread, mean_spread_stat_err} = parseResultsTable(allRows);
+    <script>
+        function makeplot() {
+            Plotly.d3.csv('<? echo $selected_season_url; ?>', (allRows) => {
+                const {
+                    point_name,
+                    energy_point,
+                    mean_energy,
+                    mean_energy_stat_err,
+                    mean_energy_sys_err,
+                    mean_spread,
+                    mean_spread_stat_err
+                } = parseResultsTable(allRows);
                 makeMeanEnergyPlot(energy_point, mean_energy, mean_energy_stat_err, point_name, "mean_energy_plot");
                 makeSpreadPlot(mean_energy, mean_spread, mean_spread_stat_err, point_name, "spread_plot");
                 makeMeanDevPlot(mean_energy, mean_energy_sys_err, point_name, "energy_dev_plot");
                 makeImpactDevPlot(point_name, mean_spread, mean_energy_sys_err, "impact_energy_dev_plot");
-            }
-            );
+            });
         }
-        
-        function parseResultsTable(allRows){
+
+        function parseResultsTable(allRows) {
             // Extracts a row following the energy point in the total csv file (or null if energy point is not exists)
             let data = {};
-            for(let key of Object.keys(allRows[0])){
+            for (let key of Object.keys(allRows[0])) {
                 data[key] = [];
             }
             data['point_name'] = [];
-            
+
             let float_keys = ['energy_point', 'mean_energy', 'mean_energy_stat_err', 'mean_spread', 'mean_spread_stat_err', 'mean_energy_sys_err'];
-            
-            for (var i=0; i<allRows.length; i++){
-                for(let key of Object.keys(allRows[i])){
+
+            for (var i = 0; i < allRows.length; i++) {
+                for (let key of Object.keys(allRows[i])) {
                     let value = allRows[i][key];
-                    if (float_keys.includes(key)){
+                    if (float_keys.includes(key)) {
                         value = parseFloat(value);
                     }
                     data[key].push(value);
                 }
-                data['point_name'].push(allRows[i]['energy_point']+'_'+allRows[i]['first_run']);
+                data['point_name'].push(allRows[i]['energy_point'] + '_' + allRows[i]['first_run']);
             }
             return data;
         }
-        
-        function linear_coeffs(x, y, yerr, remove_outliers=false){
+
+        function linear_coeffs(x, y, yerr, remove_outliers = false) {
             // Returns a chi square minimizer for linear function
-            function chi2min(X){
-                let k = X[0], b = X[1];
+            function chi2min(X) {
+                let k = X[0],
+                    b = X[1];
                 let chi2 = 0;
-                for(let i=0; i<x.length; i++){
-                    if (remove_outliers&&(y[i]<0.06))
+                for (let i = 0; i < x.length; i++) {
+                    if (remove_outliers && (y[i] < 0.06))
                         continue;
-                    y0 = k*x[i] + b;
-                    chi2 += Math.pow((y[i] - y0)/yerr[i], 2);
+                    y0 = k * x[i] + b;
+                    chi2 += Math.pow((y[i] - y0) / yerr[i], 2);
                 }
                 return chi2;
             }
             return chi2min;
         }
-        
-        function getPlotlyDefaults(){
+
+        function getPlotlyDefaults() {
             var layout = {
                 margin: {
                     l: 50,
@@ -166,19 +162,20 @@ $selected_season = substr($selected_csv, 0, -4);
             var config = {
                 toImageButtonOptions: {
                     format: 'png', // one of png, svg, jpeg, webp
-                    filename: 'spread_plot_<?echo $selected_season;?>',
+                    filename: 'spread_plot_<? echo $selected_season; ?>',
                     scale: 4,
-                  },
+                },
                 responsive: true,
                 modeBarButtonsToRemove: ['select2d', 'lasso2d', 'zoomIn2d', 'zoomOut2d', 'resetScale2d', 'hoverClosestGl2d', 'hoverClosestPie', 'toggleHover', 'toggleSpikelines'],
                 displayModeBar: true,
             };
             return {
-                "layout" : layout, "config" : config,
+                "layout": layout,
+                "config": config,
             };
         }
-        
-        function makeMeanEnergyPlot(energy_point, mean_energy, mean_energy_stat_err, texts, elementId){
+
+        function makeMeanEnergyPlot(energy_point, mean_energy, mean_energy_stat_err, texts, elementId) {
             // Plots mean energy - target energy vs target energy
             var energy_diff = [];
             Plotly.d3.zip(mean_energy, energy_point).forEach(e => energy_diff.push(e[0] - e[1]));
@@ -203,16 +200,19 @@ $selected_season = substr($selected_csv, 0, -4);
                 type: "scatter",
             };
             var traces = [trace1];
-            
-            var {layout, config} = getPlotlyDefaults();
+
+            var {
+                layout,
+                config
+            } = getPlotlyDefaults();
             layout.xaxis.title = "Target energy, MeV";
             layout.yaxis.title = "Mean energy - Target energy, MeV";
-            
+
             Plotly.newPlot(elementId, traces, layout, config);
-            
+
         }
-        
-        function makeMeanDevPlot(mean_energy, mean_energy_sys_err, texts, elementId){
+
+        function makeMeanDevPlot(mean_energy, mean_energy_sys_err, texts, elementId) {
             // Plots mean energy deviations vs mean measured energy
             var trace1 = {
                 x: mean_energy,
@@ -227,19 +227,22 @@ $selected_season = substr($selected_csv, 0, -4);
                 showlegend: false,
             };
             var traces = [trace1];
-            
-            var {layout, config} = getPlotlyDefaults()
-            
+
+            var {
+                layout,
+                config
+            } = getPlotlyDefaults()
+
             layout.yaxis.title = "Mean energy deviation, MeV";
             Plotly.newPlot(elementId, traces, layout, config);
-            
+
         }
-        
-        function makeSpreadPlot(mean_energy, mean_spread, mean_spread_stat_err, texts, elementId){
+
+        function makeSpreadPlot(mean_energy, mean_spread, mean_spread_stat_err, texts, elementId) {
             // Plots spreads vs mean measured energy
             var chi2min = linear_coeffs(mean_energy, mean_spread, mean_spread_stat_err, true);
-            var solution = fmin.nelderMead(chi2min, [0.001, 0]);       
-            
+            var solution = fmin.nelderMead(chi2min, [0.001, 0]);
+
             var trace1 = {
                 x: mean_energy,
                 y: mean_spread,
@@ -261,45 +264,52 @@ $selected_season = substr($selected_csv, 0, -4);
             };
             var trace2 = {
                 x: [Plotly.d3.min(mean_energy), Plotly.d3.max(mean_energy)],
-                y: [solution.x[0]*Plotly.d3.min(mean_energy) + solution.x[1], solution.x[0]*Plotly.d3.max(mean_energy) + solution.x[1]],
+                y: [solution.x[0] * Plotly.d3.min(mean_energy) + solution.x[1], solution.x[0] * Plotly.d3.max(mean_energy) + solution.x[1]],
                 mode: 'lines',
-                name: 'Fit: y = ' + Math.round(parseFloat(solution.x[0])*1e5)/1e5 + 'x ' + (Math.sign(solution.x[1])>0 ? "+" : "") + Math.round(solution.x[1]*1e3)/1e3,
+                name: 'Fit: y = ' + Math.round(parseFloat(solution.x[0]) * 1e5) / 1e5 + 'x ' + (Math.sign(solution.x[1]) > 0 ? "+" : "") + Math.round(solution.x[1] * 1e3) / 1e3,
                 hovermode: false,
                 hoverinfo: 'none',
             };
             var traces = [trace1, trace2];
-            
-            var {layout, config} = getPlotlyDefaults()
-            
+
+            var {
+                layout,
+                config
+            } = getPlotlyDefaults()
+
             Plotly.newPlot(elementId, traces, layout, config);
-            
+
         }
-        
-        function makeImpactDevPlot(point_name, mean_spread, mean_energy_sys_err, elementId){
+
+        function makeImpactDevPlot(point_name, mean_spread, mean_energy_sys_err, elementId) {
             let arr = Plotly.d3.zip(mean_spread, mean_energy_sys_err);
             let impact_factor = [];
-            arr.forEach(e => impact_factor.push(e[1]/Math.pow(e[0]**2+e[1]**2, 0.5)));
+            arr.forEach(e => impact_factor.push(e[1] / Math.pow(e[0] ** 2 + e[1] ** 2, 0.5)));
             var data = [{
                 type: 'bar',
                 x: impact_factor,
                 y: point_name,
                 orientation: 'h',
                 showlegend: false,
-                text: impact_factor.map(e => String(Math.round(e*1e2)) + '%'),
+                text: impact_factor.map(e => String(Math.round(e * 1e2)) + '%'),
                 textposition: 'auto',
                 hovertemplate: "Point: %{y}<br>Impact of %{text} <extra></extra>",
                 //hoverinfo: 'none',
             }];
-        
-            var {layout, config} = getPlotlyDefaults();
+
+            var {
+                layout,
+                config
+            } = getPlotlyDefaults();
             layout.xaxis.range = [0, 1];
             layout.yaxis.title = "";
             layout.xaxis.title = "Impact of mean energy deviations";
             Plotly.newPlot(elementId, data, layout, config);
         }
-        
+
         makeplot();
-</script>
+    </script>
 
 </body>
-</html>
+
+</html>

+ 7 - 0
src/scripts/README.md

@@ -0,0 +1,7 @@
+В этой директории лежат bash-скрипты, необходимые для запуска пайплайна (включается согласно `cron`)
+
+### [`compton_runs_script.sh`](./compton_runs_script.sh)
+Основной скрипт, запускающий процесс усреднения энергий.
+* запускает [compton_filter.py](../avg/compton_filter.py)
+* запускает [compton_combiner.py](../avg/compton_combiner.py)
+* обновляет соответствующие репозитории, доступные в [gogs-интерфейсе](https://cmd.inp.nsk.su/~compton/)

+ 37 - 0
src/scripts/compton_runs_script.sh

@@ -0,0 +1,37 @@
+# A script that filters compton measurements from slowdb and fills up calibration db
+# Also, this script averages compton measurements by energy points.
+
+HOMEDIR=/home/compton
+VENV=$HOMEDIR/workspace/py3.slcmd.compton_runs/bin/activate
+SOURCE_FOLDER=$HOMEDIR/workspace/work_repos/sources
+TABLES_FOLDER=$HOMEDIR/workspace/work_repos/tables
+SOURCE_BRANCH=master
+TABLES_BRANCH=online
+
+SEASON=PHI2024
+SLOWDB_TABLE=cmd3_2024_1
+COMPTON_FILTER=$SOURCE_FOLDER/src/avg/compton_filter.py
+COMPTON_COMBINER=$SOURCE_FOLDER/src/avg/compton_combiner.py
+
+# Sync repositories
+git -C $SOURCE_FOLDER checkout $SOURCE_BRANCH
+git -C $SOURCE_FOLDER pull origin $SOURCE_BRANCH
+
+git -C $TABLES_FOLDER checkout $SOURCE_BRANCH
+git -C $TABLES_FOLDER pull origin $SOURCE_BRANCH
+
+# Update averaging
+source $VENV
+python $COMPTON_FILTER --season $SLOWDB_TABLE --config database.ini --update
+mkdir -p $TABLES_FOLDER/energy_points/$SEASON
+python $COMPTON_COMBINER -s $SEASON -c database.ini --csv_dir $TABLES_FOLDER --clbrdb --only_last --energy_point_csv_folder $TABLES_FOLDER/energy_points/$SEASON
+
+# Push changes
+cd $TABLES_FOLDER
+git add ./*.csv
+git add ./*/*.csv
+git commit -m "Auto update $(date +%Y/%m/%d\ %T)"
+git push origin $TABLES_BRANCH
+cd $HOMEDIR
+
+deactivate