[lttng-dev] LTTNG-ust detailed architecture?

[Scheler, Fabian (CT RDA IOT SES-DE)]

Hi,

we would like to trace real-time applications and we consider LTTNG-ust
as one possible way to do this. But before we go for LTTNG-ust, we need
to estimate the impact LTTNG-ust could have on the instrumented
application. While I consider the user-manual of LTTNG-ust to be really
great, I am missing a more detailed architecture documentation, that
would be really helpful to answer this question.

Having a look at the picture here

https://lttng.org/docs/v2.10/#doc-plumbing

I want to review these component interactions more closely:

1. Session daemon -> liblttng-ust

This library seems to create a new thread in the context of the
instrumented application. How could this thread impact the run-time
behavior of the instrumented application, when it comes to

- submitting events to the buffer

* Which component controls/instantiates/maintains the buffer?
* How is the buffer synchronized?

- setting event-rules (i.e. enabling/disabling events)
- starting tracing sessions
- ...

* What does happen along the red dashed arrow
"Session Daemon -> liblttng-ust"?

2. liblttng-ust -> Consumer daemon

At some point the consumer daemon reads the events stored in the buffer
and writes them to some output-file.

* What does happen along the violet arrow
"liblttng-ust -> Consumer daemon"?
* Could the instrumented application be affected here somehow?

Is this documented somewhere - maybe I simply missed it.

Ciao
Fabian
--
With best regards,
Dr. Fabian Scheler

Siemens AG
Corporate Technology
Research and Development for Digitalization and Automation
Smart Embedded Systems
CT RDA IOT SES-DE
Guenther-Scharowsky-Str. 1
91058 Erlangen, Germany
Tel.: [phone]
Mobile: [phone]
mailto:***@siemens.com

www.siemens.com/ingenuityforlife

Siemens Aktiengesellschaft: Chairman of the Supervisory Board: Jim
Hagemann Snabe; Managing Board: Joe Kaeser, Chairman, President and
Chief Executive Officer; Roland Busch, Lisa Davis, Klaus Helmrich,
Janina Kugel, Cedrik Neike, Michael Sen, Ralf P. Thomas; Registered
offices: Berlin and Munich, Germany; Commercial registries: Berlin
Charlottenburg, HRB 12300, Munich, HRB 6684; WEEE-Reg.-No. DE 23691322

[Michel Dagenais]

We have been using LTTng-UST for hard real-time systems for a while. Once tracing is setup, there are few interactions between the traced application and the rest. The tracing code simply fills the sub-buffers and switches to another sub-buffer when the current one is full. In one mode, the consumer daemon polls the state of tracing sub-buffers to know when they are full and can be copied and emptied; the traced application does not have to issue any potentially blocking system call. In the default mode, the traced application sends a short message on a pipe to the consumer daemon to signal when a sub-buffer is ready to be consumed.

Post by Scheler, Fabian (CT RDA IOT SES-DE)
Hi,
we would like to trace real-time applications and we consider LTTNG-ust
as one possible way to do this. But before we go for LTTNG-ust, we need
to estimate the impact LTTNG-ust could have on the instrumented
application. While I consider the user-manual of LTTNG-ust to be really
great, I am missing a more detailed architecture documentation, that
would be really helpful to answer this question.
Having a look at the picture here
https://lttng.org/docs/v2.10/#doc-plumbing
1. Session daemon -> liblttng-ust
This library seems to create a new thread in the context of the
instrumented application. How could this thread impact the run-time
behavior of the instrumented application, when it comes to
- submitting events to the buffer
* Which component controls/instantiates/maintains the buffer?
* How is the buffer synchronized?
- setting event-rules (i.e. enabling/disabling events)
- starting tracing sessions
- ...
* What does happen along the red dashed arrow
"Session Daemon -> liblttng-ust"?
2. liblttng-ust -> Consumer daemon
At some point the consumer daemon reads the events stored in the buffer
and writes them to some output-file.
* What does happen along the violet arrow
"liblttng-ust -> Consumer daemon"?
* Could the instrumented application be affected here somehow?
Is this documented somewhere - maybe I simply missed it.
Ciao
Fabian
--
With best regards,
Dr. Fabian Scheler
Siemens AG
Corporate Technology
Research and Development for Digitalization and Automation
Smart Embedded Systems
CT RDA IOT SES-DE
Guenther-Scharowsky-Str. 1
91058 Erlangen, Germany
Tel.: [phone]
Mobile: [phone]
www.siemens.com/ingenuityforlife
Siemens Aktiengesellschaft: Chairman of the Supervisory Board: Jim
Hagemann Snabe; Managing Board: Joe Kaeser, Chairman, President and
Chief Executive Officer; Roland Busch, Lisa Davis, Klaus Helmrich,
Janina Kugel, Cedrik Neike, Michael Sen, Ralf P. Thomas; Registered
offices: Berlin and Munich, Germany; Commercial registries: Berlin
Charlottenburg, HRB 12300, Munich, HRB 6684; WEEE-Reg.-No. DE 23691322
_______________________________________________
lttng-dev mailing list
https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

[Scheler, Fabian (CT RDA IOT SES-DE)]

Hi,

thanks for your answer!

Post by Michel Dagenais
We have been using LTTng-UST for hard real-time systems for a while.
Once tracing is setup, there are few interactions between the traced
application and the rest. The tracing code simply fills the
sub-buffers and switches to another sub-buffer when the current one
is full. In one mode, the consumer daemon polls the state of tracing
sub-buffers to know when they are full and can be copied and emptied;
the traced application does not have to issue any potentially
blocking system call. In the default mode, the traced application
sends a short message on a pipe to the consumer daemon to signal when
a sub-buffer is ready to be consumed.

In principle, this sounds quite well, but there are still plenty of
other thinks we would like to know in more detail:

- How does the ust_listener_thread interact with the instrumented
application?
- How does the ust_listener_thread interact with the consumderd (I guess
your answer targeted this part, right?) and/or the sessiond?
- What is the purpose of the shared memory in the ringbuffer (shm.{c,h})?
- How does the consumerd poll the buffers?
- ...

So, it would be really great if there is some document where this
architecture is noted down. Maybe, there are also helpful comments in
the code - if you can point me there, this would also be helpful.

Ciao
Fabian

Post by Michel Dagenais
----- Le 24 Oct 18, à 6:53, Scheler, Fabian (CT RDA IOT SES-DE)

Post by Scheler, Fabian (CT RDA IOT SES-DE)
Hi,
we would like to trace real-time applications and we consider
LTTNG-ust as one possible way to do this. But before we go for
LTTNG-ust, we need to estimate the impact LTTNG-ust could have on
the instrumented application. While I consider the user-manual of
LTTNG-ust to be really great, I am missing a more detailed
architecture documentation, that would be really helpful to answer
this question.
Having a look at the picture here
https://lttng.org/docs/v2.10/#doc-plumbing
1. Session daemon -> liblttng-ust
This library seems to create a new thread in the context of the
instrumented application. How could this thread impact the
run-time behavior of the instrumented application, when it comes
to
- submitting events to the buffer
* Which component controls/instantiates/maintains the buffer? * How
is the buffer synchronized?
- setting event-rules (i.e. enabling/disabling events) - starting
tracing sessions - ...
* What does happen along the red dashed arrow "Session Daemon ->
liblttng-ust"?
2. liblttng-ust -> Consumer daemon
At some point the consumer daemon reads the events stored in the
buffer and writes them to some output-file.
* What does happen along the violet arrow "liblttng-ust -> Consumer
daemon"? * Could the instrumented application be affected here
somehow?
Is this documented somewhere - maybe I simply missed it.
Ciao Fabian -- With best regards, Dr. Fabian Scheler
Siemens AG Corporate Technology Research and Development for
Digitalization and Automation Smart Embedded Systems CT RDA IOT
SES-DE Guenther-Scharowsky-Str. 1 91058 Erlangen, Germany Tel.: +49
1522 1702973 Mobile: [phone]
www.siemens.com/ingenuityforlife
Siemens Aktiengesellschaft: Chairman of the Supervisory Board: Jim
Hagemann Snabe; Managing Board: Joe Kaeser, Chairman, President
and Chief Executive Officer; Roland Busch, Lisa Davis, Klaus
Helmrich, Janina Kugel, Cedrik Neike, Michael Sen, Ralf P. Thomas;
Registered offices: Berlin and Munich, Germany; Commercial
registries: Berlin Charlottenburg, HRB 12300, Munich, HRB 6684;
WEEE-Reg.-No. DE 23691322
_______________________________________________ lttng-dev mailing
https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

--
With best regards,
Dr. Fabian Scheler

Siemens AG
Corporate Technology
Research and Development for Digitalization and Automation
Smart Embedded Systems
CT RDA IOT SES-DE
Guenther-Scharowsky-Str. 1
91058 Erlangen, Germany
Tel.: [phone]
Mobile: [phone]
mailto:***@siemens.com

www.siemens.com/ingenuityforlife

Siemens Aktiengesellschaft: Chairman of the Supervisory Board: Jim
Hagemann Snabe; Managing Board: Joe Kaeser, Chairman, President and
Chief Executive Officer; Roland Busch, Lisa Davis, Klaus Helmrich,
Janina Kugel, Cedrik Neike, Michael Sen, Ralf P. Thomas; Registered
offices: Berlin and Munich, Germany; Commercial registries: Berlin
Charlottenburg, HRB 12300, Munich, HRB 6684; WEEE-Reg.-No. DE 23691322