安裝和設定

https://docs.nvidia.com/tao/tao-toolkit/text/tao_toolkit_quick_start_guide.html#run-sample-jupyter-notebooks

Jupyter notebook改成暗色系

1
2
3

pip install jupyterthemes

jt -t monokai -f fira -fs 10 -nf ptsans -nfs 11 -N -kl -cursw 2 -cursc r -cellw 95% -T

重新啟動jupyter notebook

https://www.kaggle.com/getting-started/97540#983780

search

使用keyword使得搜尋條件必須完全符合

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

GET people-2023.04/_search
{ "query":
  { 
    "bool": {
      "must": [
        {
          "range": {
            "@timestamp": {
              "gt" : "2023-04-08T10:22:28Z",
              "lt" : "2023-04-08T10:22:44Z"
            }
          }
        },
        {
          "match": {
            "StreamID.keyword": "eb2baea2-d52c-434c-af44-256c0301f4df"
          }
        }
      ]
    }
  } 
}

delete

1
2
3
4
5
6
7
8
9
10

POST poc/_delete_by_query
{
    "query":{
        "range":{
            "@timestamp":{  
              "gt" : "2018-05-04T23:04:00Z"
              }
        }
    }
}

CH1

• 樹梅派不適合做real-time的應用
• 樹梅派不適合作為production的應用，如果要做成production的應用，可以考慮BeagleBone
• 其他參考資源:
  • www.elinux.org
  • www.exploringrpi.com
  • www.derekmolloy.ie
• 建議購買的配件
  • USB to Serial UART TTL 3.3 V (for Finding Problems)線，可以用於透過USB就可以使用command line控制樹梅派

設定檔編寫

logstash基本元素

input, filter, 和 output，每個元素可能一個或多個

建立一個基本的pipeline

下面建立一個基本的pipeline，從stdin讀取資料，然後輸出到stdout。我們將建立一個first-pipeline.conf並且放在C:\ELK\logstash-7.17.3\logstash-7.17.3(與bin同一個資料夾)

1
2
3
4
5
6
7
8
9
10
11
12
13

# The # character at the beginning of a line indicates a comment. Use
# comments to describe your configuration.
input {
stdin { }
}
# The filter part of this file is commented out to indicate that it is
# optional.
# filter {
#
# }
output {
stdout {}
}

檢查pipeline語法

打開powershell，利用以下指令檢查pipeline語法，注意要確認沒有任何錯誤，因為檢查程式如果找不到檔案位置最後也會顯示檢查OK。
我們在C:\ELK\logstash-7.17.3\logstash-7.17.3輸入以下指令bin/logstash -f first-pipeline1.conf --config.test_and_exit

輸出如下

1
2
3
4
5
6
7
8
9
10
11

Using LS_JAVA_HOME defined java: C:\ELK\logstash-7.17.3\logstash-7.17.3\jdk\
WARNING: Using LS_JAVA_HOME while Logstash distribution comes with a bundled JDK.
OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
Sending Logstash logs to C:/ELK/logstash-7.17.3/logstash-7.17.3/logs which is now configured via log4j2.properties
[2023-05-05T14:24:31,585][INFO ][logstash.runner          ] Log4j configuration path used is: C:\ELK\logstash-7.17.3\logstash-7.17.3\config\log4j2.properties
[2023-05-05T14:24:31,601][INFO ][logstash.runner          ] Starting Logstash {"logstash.version"=>"7.17.3", "jruby.version"=>"jruby 9.2.20.1 (2.5.8) 2021-11-30 2a2962fbd1 OpenJDK 64-Bit Server VM 11.0.14.1+1 on 11.0.14.1+1 +indy +jit [mswin32-x86_64]"}
[2023-05-05T14:24:31,601][INFO ][logstash.runner          ] JVM bootstrap flags: [-Xms1g, -Xmx2g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djruby.compile.invokedynamic=true, -Djruby.jit.threshold=0, -Djruby.regexp.interruptible=true, -XX:+HeapDumpOnOutOfMemoryError, -Djava.security.egd=file:/dev/urandom, -Dlog4j2.isThreadContextMapInheritable=true]
[2023-05-05T14:24:31,761][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
[2023-05-05T14:24:32,816][INFO ][org.reflections.Reflections] Reflections took 78 ms to scan 1 urls, producing 119 keys and 419 values
Configuration OK
[2023-05-05T14:24:33,781][INFO ][logstash.runner          ] Using config.test_and_exit mode. Config Validation Result: OK. Exiting Logstash

自動重載設定檔

檢查通過後我們用--config.reload.automatic選項重新載入設定檔，如此一來就不用一直重啟程式。在過程中你可能會看到忽略pipelines.yml的警告，因為我們已經在命令中明確指定要用的設定檔了，所以可以忽略這個警告，之後我們再學習使用pipelines.yml

bin/logstash -f first-pipeline.conf --config.reload.automatic

等到看到[main] Pipeline started {"pipeline.id"=>"main"}這個訊息後，我們可以直接輸入一些文字，按下enter，就可以看到輸入的文字被輸出到stdout了。
例如入Hello wordl後按下Enter，結果如下

1
2
3
4
5
6
7

Hello World
{
      "@version" => "1",
          "host" => "eastcoastVM01",
       "message" => "Hello World\r",
    "@timestamp" => 2023-05-05T06:32:33.396Z
}

使用 Grok Filter 外掛程式(Plugin)

Grok是眾多logstash外掛程式的其中一個，這裡可以看到更多關於logstash的外掛程式。

grok filter plugin可以幫我們將沒有結構化的log轉化成結構化的log

多個資料來源

以下範例是多個資料來源，分別來自twitter和firebeat，輸出也有多個，分別為elasticsearch和寫到檔案

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

input {
    twitter {
        consumer_key => "enter_your_consumer_key_here"
        consumer_secret => "enter_your_secret_here"
        keywords => ["cloud"]
        oauth_token => "enter_your_access_token_here"
        oauth_token_secret => "enter_your_access_token_secret_here"
    }
    beats {
        port => "5044"
    }
}
output {
    elasticsearch {
        hosts => ["IP Address 1:port1", "IP Address 2:port2", "IP Address 3"]
    }
    file {
        path => "/path/to/target/file"
    }
}

建立一個接收rabbitmq訊號的範例

解析rabbitmq訊號

在input中我們可以加入input plugin，這裡我們使用rabbitmq plugin，先建立一個最簡單的範例並且將解析結果輸出到console。
在rabbitmq plugin中的說明文件有提到，預設的輸出是json codec

而設定rabbitmq的參數在這裡，我們會需要設定rabbitmq的連線資訊，範例如下。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

# The # character at the beginning of a line indicates a comment. Use
# comments to describe your configuration.
input {
  rabbitmq {
    host => "localhost"
    port => 5672
    username => guest
    password => guest
  }
}
# The filter part of this file is commented out to indicate that it is
# optional.
# filter {
#
# }
output {
stdout {}
}

抽取事件中的欄位

有時候我們會需要依據事件中欄位的資料做一些處理，Field Reference可以幫助我們抽取事件中的欄位。以下事件為範例
，如果要取得第一階的欄位如agent, ip, request, response, ua，只需要一個[]就可以取得例如[request]，如果是第二階欄位如os則要用[ua][os]

1
2
3
4
5
6
7
8
9
10
11
12

{
  "agent": "Mozilla/5.0 (compatible; MSIE 9.0)",
  "ip": "192.168.24.44",
  "request": "/index.html"
  "response": {
    "status": 200,
    "bytes": 52353
  },
  "ua": {
    "os": "Windows 7"
  }
}

對欄位操作Mutate filter plugin

https://www.elastic.co/guide/en/logstash/7.17/plugins-filters-mutate.html#plugins-filters-mutate-replace

檢查子欄位是否存在

https://github.com/elastic/logstash/issues/10215#issuecomment-447912618

加入一個filed，其值是一個sub field

https://stackoverflow.com/a/39126309

測試watchdog

L4T已經預設有watchdog，可以透過下面指令測試，他預設的機制是如果有人讀取/dev/watchdog這個檔案，他就會開始倒數計時，如果有人寫入檔案，則timer重置，下面指令將會讓watchgod重啟系統。

1

sudo tail -f /dev/watchdog

如果要避免重啟就必須寫入/dev/watchdog檔。或是結束tail -f

https://forums.developer.nvidia.com/t/configuring-watchdog-timer-on-tx1/44361/2?u=jenhao

參考:
官方文件下載區
https://developer.nvidia.com/embedded/downloads

注意是否有從C++ 呼叫C語言的函式，如果有必須要使用extern C

https://hackmd.io/@rhythm/HyOxzDkmD

https://www.airs.com/blog/archives/38

https://www.google.com/search?q=Linkers+part+site%3Ahttps%3A%2F%2Fwww.airs.com%2F&rlz=1C1GCEU_zh-TWTW902TW902&oq=Linkers+part+site%3Ahttps%3A%2F%2Fwww.airs.com%2F&gs_lcrp=EgZjaHJvbWUyBggAEEUYOdIBCDM1ODlqMGo3qAIAsAIA&sourceid=chrome&ie=UTF-8#ip=1

注意function是否為static function

c語言中的static function在function被定義的檔案以外的scope是看不到的

undefined reference to

表示必要的函式庫或是.o檔沒有被連接

1. 用nm指令確認函式確實存在.so檔裡面例如:
   nm -D libnvds_utils.so --defined-only的輸出如下，這個指令只會列出有對外開放的函式

   1 2 3 4 5 6

   0000000000001470 T nvds_dependencies_version_print 00000000000018d0 T nvds_mask_utils_resize_to_binary_argb32 00000000000013f0 T nvds_version 0000000000001410 T nvds_version_print 0000000000001310 T _Z18libnvds_utils_initv 0000000000001300 T _Z20libnvds_utils_deinitv

   https://stackoverflow.com/a/4514781

• nm不只可以用在函式庫檔，.o檔也可以用
  nm deepstream_source_bin.o --defined-only

  1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17

  0000000000000000 b install_mux_eosmonitor_probe 0000000000000010 b last_reset_time_global.31491 0000000000000008 b mutex.31460 00000000000030a8 t nvstreammux_eosmonitor_probe_func 0000000000006ab0 T reset_encodebin 00000000000065e8 T reset_source_pipeline 000000000000133c t restart_stream_buf_prob 0000000000002ffc t rtspsrc_monitor_probe_func 0000000000001270 t seek_decode 0000000000000020 t set_camera_csi_params 00000000000000bc t set_camera_v4l2_params 0000000000006a04 T set_source_to_playing 0000000000002178 t smart_record_callback 0000000000002270 t smart_record_event_generator 0000000000002e74 t watch_source_async_state_change 00000000000026d0 t watch_source_status ...

/usr/bin/ld: cannot find …

編譯過程中發現ld找不到某個函式庫例如/usr/bin/ld: cannot find -ljpeg，可以利用指令ld <函式庫> --verbose可以查看ld找了那些路徑，例如目前ld找不到-ljpeg可以利用ld -ljpeg --verbose，輸出如下

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

ld: mode aarch64linux
attempt to open /usr/local/lib/aarch64-linux-gnu/libjpeg.so failed
attempt to open /usr/local/lib/aarch64-linux-gnu/libjpeg.a failed
attempt to open /lib/aarch64-linux-gnu/libjpeg.so failed
attempt to open /lib/aarch64-linux-gnu/libjpeg.a failed
attempt to open /usr/lib/aarch64-linux-gnu/libjpeg.so failed
attempt to open /usr/lib/aarch64-linux-gnu/libjpeg.a failed
attempt to open /usr/local/lib/libjpeg.so failed
attempt to open /usr/local/lib/libjpeg.a failed
attempt to open /lib/libjpeg.so failed
attempt to open /lib/libjpeg.a failed
attempt to open /usr/lib/libjpeg.so failed
attempt to open /usr/lib/libjpeg.a failed
attempt to open /usr/aarch64-linux-gnu/lib/libjpeg.so failed
attempt to open /usr/aarch64-linux-gnu/lib/libjpeg.a failed
ld: cannot find -ljpeg

注意makefile的擺放順序

連接器指令放最後面
例如

1

gcc -I/usr/local/include -o yaml_reader yaml_reader.c  -L/usr/local/lib -lfyaml

https://stackoverflow.com/questions/22426574/gcc-undefined-reference-to

https://stackoverflow.com/questions/16710047/usr-bin-ld-cannot-find-lnameofthelibrary

注意cmake是否有把程式加入編譯

add_executable(…)內加入的cpp檔會被編譯，確認是否有將程式加入

下載tracetcp

https://github.com/0xcafed00d/tracetcp

檢查遠端伺服器有沒有開port

1

nc -z -v -w5 <host> <port>

https://stackoverflow.com/a/9463554

1. NetworkManager TUI，nmtui設定網路ip
   安裝

   1 2	sudo apt update sudo apt install network-manager -y

參考:
https://ithelp.ithome.com.tw/m/articles/10267179

2. 資料夾空間查詢TUI

• ncdu

sudo apt install ncdu
https://ephrain.net/mac-%E5%9C%A8%E7%B5%82%E7%AB%AF%E6%A9%9F%E4%B8%AD%E4%BD%BF%E7%94%A8-ncdu%EF%BC%8C%E6%AA%A2%E8%A6%96%E7%A1%AC%E7%A2%9F%E7%A9%BA%E9%96%93%E5%A4%A7%E5%B0%8F/

https://github.com/rofl0r/ncdu

• dua-cli

https://github.com/Byron/dua-cli?tab=readme-ov-file

• duc
  https://duc.zevv.nl/

1. 遠端桌面
   Nomachine

2. 硬體狀態jtop

   1 2 3

   sudo apt update sudo apt install python3-pip sudo pip3 install -U jetson-stats

3. 關閉GUI節省GPU資源

• 暫時關閉以及打開方法

  1	sudo init 3 #暫時關閉

1

sudo init 5  #打開

• 重啟後不再打開GUI

  1	sudo systemctl set-default multi-user.target #關閉

1

sudo systemctl set-default multi-user.target #打開

參考:
https://forum.nomachine.com/topic/problems-connecting-with-nomachine-in-pc-client-without-monitor

1. 無螢幕的Server(Headless server)並且有顯卡，登入後螢幕是黑的
   https://kb.nomachine.com/AR03P00973 根據步驟1的做法

1
2

sudo systemctl stop display-manager
sudo /etc/NX/nxserver --restart

參考:
https://forum.nomachine.com/topic/problems-connecting-with-nomachine-in-pc-client-without-monitor