send clip in notification (not "only" the URL)?

[MrColumbo]

Currently I use homeassistant to receieve notfications and there i can send a clip directly to Telegram. I am wondering if this work with firagete-notify as well since it is quite nice like telegram shows a preview (like a thumbnail video) .

thanks and kind regards

[freefd]

Hi @MrColumbo,

Just side notes from yet another user

Upon discovering Telegram's integration with frigate-notify, I was also expecting to receive notifications with attached clips. But later, after having thoughts on this feature, I decided not to request it because:

1. Privacy. A clip from a private physical and network perimeter must not be published on the internet, even on Telegram.
2. Notification delivery speed. Afair, the message won't be sent until the clip is uploaded to Telegram's servers. This can be overcome by sending 2 messages: the 1st fast with the snapshot and links (the current behavior), the 2nd delayed with the clip as an attached file.
3. Attached clip size. There is no way to predict the clip size created from a camera in a particular Frigate + frigate-notify installation. For frigate-notify, this means a requirement to be a proxy for an unknown binary blob size between Frigate API and Telegram API. Impact to performance: memory consumption, handling possible network issues b/w Frigate API and frigate-notify (as the frigate-notify may be placed outside the same network), etc.

Could you please describe how the HA sends Telegram notification with clip attached: current behavior, what you would like to improve, other special notes?

Thanks.

[MrColumbo]

Hi @freefd

thanks for replying to my message and sharing your thoughts.

Well - the HA integration for whatever reason miss come notifications. There are 3 cameras capturing the same are with some overlap. When I look at frigate directly i see typically 3 events - one from each camera but somehow i often get just 2 notifications via HA. Since HA is just to used to send notifications and there i nothing else i do there with the cameras I thought it might be nice to try a different way fully independent from HA.

[freefd]

Hi @MrColumbo,

Good and strong description of the business case, but I'm asking more about the nature of HA's messaging. For example:

• Does HA send a message immediately if something appears in the camera's FOV?
• Or will it wait until the scene recording is complete, and the entire clip is recorded and ready to be sent?

Because I see some trouble there: if HA sends a message immediately while the scene is still in action, how to see the clip online while it's still in the recording stage on the Frigate side.

Dear @0x2142, FYI, Telegram has two kinds of "Video Streaming":

1. Streaming already uploaded video: https://core.telegram.org/constructor/documentAttributeVideo. Description: https://telegram.org/blog/android-streaming
2. Live video streaming over RTMP: https://core.telegram.org/method/phone.createGroupCall. Description: https://telegram.org/blog/live-streams-forwarding-next-channel. Example: https://github.com/OpenIPC/wiki/blob/master/en/howto-streaming-telegram.md

If there are expectations of seeing a clip that is still being recorded on the Frigate side, the 2nd option should be considered. But I don't remember it being possible outside of group chat.

Probably, there is the 3rd option for that I can't easily say is possible: the call a user and stream real-time video from Frigate instead of video from the user's phone/workstation camera.

Long story short: without a deep dive into the Telegram API, I'd stick with option 2 (with explicit enabler from a configuration perspective!) from #98 (comment).

[MrColumbo]

Hi @freefd

my trigger fires at the end of an event

alias: Frigate send mp4 video to Telegram
description: ""
trigger:

• platform: mqtt
  topic: frigate/events
• platform: template
  value_template: "{{ trigger.payload_json['type'] == 'end' }}"
  condition:
• condition:
  • condition: template
    value_template: "{{ trigger.payload_json['type'] == 'end' }}"
    action:
• service: telegram_bot.send_video
  data:
  authentication: digest
  url: >-
  http://192.168....:8123/api/frigate/notifications/{{trigger.payload_json["after"]["id"]}}/clip.mp4
  mode: parallel
  max: 10

....now that i am looking at it i am wondering if the condition is redundant since it is also the trigger ....

hope that helps

[freefd]

"{{ trigger.payload_json['type'] == 'end' }}"

Useful as a chocolate teapot.

Well... It seems, frigate-notify may partially behave the same way, but be more effective overall:

The user will still receive an instant message with a snapshot (steps 03 .. 07 of the sequence) as soon as something important appears in the camera's FOV and is detected by the Frigate.
Also, frigate-notify with "clip attachment" feature enabled should continue track the same event_Id for type = end, meaning that the clip recording is complete, and the clip can be sent to the user as a second Telegram message (steps 09 .. 15 of the sequence).
And, in-flight transcoding (step 13 of the sequence) can also be handy to support Video Streaming in Telegram if the clip is not AVC encoded, such as if the camera is streaming in the H.265 codec.
By the way, all my cameras were intentionally set up to stream H.264 to avoid codec issues in modern browsers due to patent wars. And just checked, files from their streams the Frigate stores as is:

$ ffprobe -v quiet -print_format json -show_streams 14.24.mp4 | jq .streams[0].codec_long_name -r
H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10

Probably, Video Streaming will work OOB for me here :)

[0x2142]

I think for the current moment, this is likely not something I will plan to add. However, I'll keep it on my list as something to potentially come back to later on.

A few of my thoughts:

• If I did support video clips for Telegram, I would want it to be available for any other notification platforms that could accept video files. So it would take some work to figure out what is needed for each one of them & any limitations.
• I would stick to something supported by all notification providers - like only sending as an attachment, rather than the live streaming option mentioned above.
• Telegram seems to have a generous attachment limit, but others are much more limited. A quick search of the providers we currently support & I think Pushover might be the smallest limit at 2.5MB, while some others are 15MB, 20MB, etc...
• I have no interest in supporting video transcoding at this time, so clips would be sent as-is.
• The new /reviews endpoint seems to automatically generate preview GIFs, which could be used instead. This might be a good middle ground since they're already being generated by Frigate. That being said, I'll need to figure out how quickly those are generated - and a quick check through some of mine show GIFs up to 2MB, so size may still be a consideration here.

[RebeliusGaming]

I just want to express my interest in this feature.

Thanks for your awesome work.

[freefd]

In-flight transcoding can also be handy to support Video Streaming in Telegram if the clip is not AVC encoded, such as if the camera is streaming in the H.265 codec.

One more step closer to the human-friendly video streaming in Telegram: https://telegram.org/blog/dynamic-video-quality-and-more