Question regarding date comparison for watch state #317

aptalca · 2023-05-26T21:50:19Z

aptalca
May 26, 2023

I just started using watchstate and am trying to figure out some of the functionality details.

I have two emby servers, one local and one remote on a VPS. The one on the VPS accesses the media synced to a Google Drive share (rclone).

My workflow:

Media first gets added to my local server and is picked up by the local Emby instance.
At night, local media gets synced to the Google Drive share and becomes available to the second Emby server on the VPS.

Will I have issues with watch state syncing if the following occurs:

Media added to local Emby.
I watch it and is marked as watched.
Watchstate runs and imports the media and the watch state as played.
At night the media gets synced to GDrive.
Remote Emby on VPS will pick up the media and add it to the library.
Watchstate runs again

At that point, will watchstate see the media as played on server 1, but unplayed on server 2, and since server 2 acquired/added the media at a later time than it was played on server 1, will it determine that the latest watch state is unplayed?

If so, that's going to be a regular problem for me.

I used to rely on syncing watch status with trakt so I could keep it in sync between 3-4 servers. But the one option I had for the apps syncing with trakt, was to let it only import/export played status. In other words, if a media is marked played on any of the servers, it will be accepted as the latest watch state no matter the dates.

The downside was, if something got accidentally marked as played, I would have to mark it as unwatched on all the servers and trakt.tv.

Is there such an option here?

Thanks

Answered by arabcoders

May 27, 2023

Hi, this topic is quite complicated, So i have to break it into few parts to explain the logic and process.

If you use webhooks to notify watchstate of the status change, we have marked itemadd events as tainted event with will not alter the state of the object. If however, you use regular import/export, it will check the date on object. if the date on the object is newer than when you watched the said item, the behavior is as expected newer > older. However, the method you use to transfer your media from local to remote could maintain the file creation date, you have to enable the option for emby to use it though Server > Library > Advanced > Date added behavior for new content: > Use fi…

View full answer

arabcoders · 2023-05-27T18:24:49Z

arabcoders
May 27, 2023
Maintainer

Hi, this topic is quite complicated, So i have to break it into few parts to explain the logic and process.

If you use webhooks to notify watchstate of the status change, we have marked itemadd events as tainted event with will not alter the state of the object. If however, you use regular import/export, it will check the date on object. if the date on the object is newer than when you watched the said item, the behavior is as expected newer > older. However, the method you use to transfer your media from local to remote could maintain the file creation date, you have to enable the option for emby to use it though Server > Library > Advanced > Date added behavior for new content: > Use file creation date . Other than that unfortunately there is no way to know when you really marked item as watched or not. If you do not use the remote emby you could mark the backend as metadata only thus it will not alter your status.

I personally use and recommend the webhook method. it will sync your state much faster than regular import/export. this doesn't mean you dont use the import/export you can enable both.

The downside was, if something got accidentally marked as played, I would have to mark it as unwatched on all the servers and trakt.tv.

With watchstate you only need to mark it on one server and it will propagate to the other servers.

You can always backup your play state using state:backup command. it helps to keep backups incase you screw things up =) and it's not really big and can be used to restore the state to any supported backend.

0 replies

aptalca · 2023-05-28T20:46:09Z

aptalca
May 28, 2023
Author

Unfortunately Rclone to Google Drive does not preserve the file creation date.

I will rely on webhooks then, that's great.
But if I ever do a regular import/export later on, it will change the watch state to match the latest date, and potentially revert watched status back to unplayed, is that right?

1 reply

arabcoders May 28, 2023
Maintainer

From memory, No it won't change after the item had been added via webhook if i remember correctly. You can always try it with one movie for example and see if it's gets marked as unplayed when it has been added to the remote source.

aptalca · 2023-05-29T13:31:44Z

aptalca
May 29, 2023
Author

I tested it, but I'm observing some weird behavior that I can't figure out.

It turns out, rclone Google Drive sync does set the file creation date, so the addition time on the remote emby matches the local emby. However, I also have a couple of plex servers (one local, one remote) and they go by when it was actually scanned into the library so that won't help there.

In any case, here's the weird behavior:

Added the new Blacklist episode to the local Emby, marked it as played.
Webhook marked it as played in watchstate as can be seen in db:list. All good so far.
Later that night, it got synced to Google Drive and scanned into the remote emby and plex servers.
In the morning, I did an export, no export of blacklist watchstate occurred.
I did an import, watchstate imported the blacklist episode addition to the 2 plex servers and the remote emby server.
Did another export and watchstate says there are no play state changes since last export.

Now I have the episode marked as played on the lcoal emby and watchstate, but is is unplayed on the other 3 servers.
The main emby is enabled for import/export. The other 3 are only enabled for export and metadata only import.

Did it get messed up because I did the export before import?

0 replies

arabcoders · 2023-05-29T17:52:00Z

arabcoders
May 29, 2023
Maintainer

Let's define the scenario first.

local emby marked item "blacklist 1x1" as played on 2023-05-29 15:00.
late at night the items get synced to remote servers.
plex does send correct item_add webhook event.
emby item_add event is kindof broken see this link

so far everything is within the expected behavior, since the item_add date on remote servers is newer than when you watched the items, it will not change the state to played. to fix this, you have to change the cron options for export to ignore the date comparison, sadly that's the downside of supporting mark as unplayed.

0 replies

aptalca · 2023-05-29T18:03:04Z

aptalca
May 29, 2023
Author

I see. But I can't get unplayed working either via webhook.

I marked a movie as played on local Emby. Webhook received, updated play state on watchstate.
Tried a state:push and it updated play state on local Plex. All good.

Then I marked it as unplayed on local Emby.
Webhook received, but it's still marked as played on watchstate

Here's the json output of the db:list

# docker exec -ti watchstate console db:list -l 1 -o json
[
    {
        "id": 13715,
        "type": "movie",
        "updated": "2023-05-29T13:24:06-04:00",
        "watched": true,
        "via": "emby",
        "title": "Spinning Gold",
        "year": 2023,
        "season": null,
        "episode": null,
        "parent": [],
        "guids": {
            "guid_imdb": "tt2071590",
            "guid_tmdb": "326316"
        },
        "metadata": {
            "emby": {
                "id": "2068671",
                "type": "movie",
                "watched": "0",
                "via": "emby",
                "title": "Spinning Gold",
                "guids": {
                    "guid_imdb": "tt2071590",
                    "guid_tmdb": "326316"
                },
                "added_at": "1685364089",
                "extra": {
                    "date": "2023-03-31"
                },
                "year": "2023",
                "path": "/mnt/user/Movies/Spinning Gold (2023)/Spinning Gold (2023) Bluray-720p ().mkv",
                "library": "54392",
                "played_at": "1685364089"
            },
            "plex": {
                "id": "253319",
                "type": "movie",
                "watched": "1",
                "via": "plex",
                "title": "Spinning Gold",
                "guids": {
                    "guid_imdb": "tt2071590",
                    "guid_tmdb": "326316",
                    "guid_tvdb": "115868"
                },
                "added_at": "1685367715",
                "extra": {
                    "date": "2023-03-31"
                },
                "library": "8",
                "year": "2023",
                "path": "/tower/Movies/Spinning Gold (2023)/Spinning Gold (2023) Bluray-720p ().mkv",
                "played_at": "1685381352"
            }
        },
        "extra": {
            "emby": {
                "event": "item.markunplayed",
                "received_at": "2023-05-29T13:49:27-04:00"
            },
            "plex": {
                "event": "task.import",
                "received_at": "2023-05-29T13:36:28-04:00"
            }
        }
    }
]

And when I try an export, I get:

# docker exec -ti watchstate console state:export -v
[2023-05-29T13:55:48-04:00] NOTICE: DATABASE: Loading changed items since [2023-05-29 13:30:07 EDT].
[2023-05-29T13:55:48-04:00] NOTICE: DATABASE: No play state change detected since [2023-05-29 13:30:07 EDT].

There are no queued webhook events. state:push -v has no output.

Now my emby and plex servers are out of sync where the movie is marked as played on plex, but unplayed on emby.

Sorry for the series of questions. There is quite a bit of a learning curve with the process and the commands so I keep digging deeper into them :-)

8 replies

arabcoders May 29, 2023
Maintainer

Alright everything seems good so far lets see why the movie isn't being marked as unplayed state:import --trace --context -vvvfs emby run this and grep the relevant part as well.

aptalca May 29, 2023
Author

[2023-05-29T17:06:05-04:00] DEBUG: Processing [emby] payload. { (library: [ (id: 54392), (title: Movies), (type: movies), (totalRecords: 3721), (url: http://emby:8096/Users/XXXXXXXXXXXXXXXX/items/?sortBy=DateCreated&sortOrder=Ascending&parentId=54392&recursive=true&enableUserData=true&enableImages=false&excludeLocationTypes=Virtual&fields=ProviderIds%2CDateCreated%2COriginalTitle%2CSeasonUserData%2CDateLastSaved%2CPremiereDate%2CProductionYear%2CPath&includeItemTypes=Movie%2CEpisode&limit=1000&startIndex=3000) ]), (segment: [ (number: 4), (of: 4), (size: 1000) ]), (body: [ (Name: Spinning Gold), (OriginalTitle: Spinning Gold), (ServerId: XXXXXXXXXXXXXXX), (Id: 2068671), (DateCreated: 2023-05-29T12:41:29.0000000Z), (PremiereDate: 2023-03-31T04:00:00.0000000Z), (Path: /mnt/user/Movies/Spinning Gold (2023)/Spinning Gold (2023) Bluray-720p ().mkv), (RunTimeTicks: 82593820000), (ProductionYear: 2023), (ProviderIds: [ (Imdb: tt2071590), (Tmdb: 326316) ]), (IsFolder: false), (Type: Movie), (UserData: [ (PlaybackPositionTicks: 0), (PlayCount: 0), (IsFavorite: false), (Played: false) ]), (MediaType: Video) ]) }
[2023-05-29T17:06:05-04:00] DEBUG: MAPPER: [emby] [Spinning Gold (2023)] metadata and play state is identical. { (id: 13715), (state: [ (database: [ (id: 13715), (type: movie), (updated: 1685381046), (watched: 1), (via: emby), (title: Spinning Gold), (year: 2023), (season: None), (episode: None), (parent: [  ]), (guids: [ (guid_imdb: tt2071590), (guid_tmdb: 326316) ]), (metadata: [ (emby: [ (id: 2068671), (type: movie), (watched: 0), (via: emby), (title: Spinning Gold), (guids: [ (guid_imdb: tt2071590), (guid_tmdb: 326316) ]), (added_at: 1685364089), (extra: [ (date: 2023-03-31) ]), (year: 2023), (path: /mnt/user/Movies/Spinning Gold (2023)/Spinning Gold (2023) Bluray-720p ().mkv), (library: 54392), (played_at: 1685364089) ]), (plex: [ (id: 253319), (type: movie), (watched: 1), (via: plex), (title: Spinning Gold), (guids: [ (guid_imdb: tt2071590), (guid_tmdb: 326316), (guid_tvdb: 115868) ]), (added_at: 1685367715), (extra: [ (date: 2023-03-31) ]), (library: 8), (year: 2023), (path: /tower/Movies/Spinning Gold (2023)/Spinning Gold (2023) Bluray-720p ().mkv), (played_at: 1685381352) ]) ]), (extra: [ (emby: [ (event: item.markunplayed), (received_at: 2023-05-29T14:32:08-04:00) ]), (plex: [ (event: task.import), (received_at: 2023-05-29T13:36:28-04:00) ]) ]) ]), (backend: [ (id: None), (type: movie), (updated: 1685364089), (watched: 0), (via: emby), (title: Spinning Gold), (year: 2023), (season: None), (episode: None), (parent: [  ]), (guids: [ (guid_imdb: tt2071590), (guid_tmdb: 326316) ]), (metadata: [ (emby: [ (id: 2068671), (type: movie), (watched: 0), (via: emby), (title: Spinning Gold), (guids: [ (guid_imdb: tt2071590), (guid_tmdb: 326316) ]), (added_at: 1685364089), (extra: [ (date: 2023-03-31) ]), (library: 54392), (year: 2023), (path: /mnt/user/Movies/Spinning Gold (2023)/Spinning Gold (2023) Bluray-720p ().mkv) ]) ]), (extra: [ (emby: [ (event: task.import), (received_at: "2023-05-29T17:06:05-04:00") ]) ]) ]) ]) }

arabcoders May 29, 2023
Maintainer

Thanks. i will look into it once i have free time, but before that could you try marking the movie as played on emby and see if the state and date changes on the db object? if it does try to mark as unplayed and see if it goes through this time. the metadata date has to be identical to that of the server for objects to be marked ad unplayed. i think you didn't reset your database when you refreshed your emby metadata correct? if so delete the database and reimport your state.

aptalca May 29, 2023
Author

Not sure if it matters, but while I'm trying to figure out the basics, I enabled metadata and state import on the local emby server, but the local plex server only does metadata import.

Yes, I did do a full refresh metadata on emby yesterday.

Actually, let me wipe watchstate fully and start over because I did mess with the libraries on all 4 servers to make things work properly. There were some mismatched items, and then the whole folder entries on the emby server debacle that led to the metadata refresh.

Now that the dbs are all synced up, I'll start over, and make better use of the debug tools I finally figured out.

Thanks so much for putting up with me :-)

arabcoders May 29, 2023
Maintainer

Thanks hopefully things go well. some of those APIs and the logic yo handle the events are crud but somehow works. In due time i would like to rewrite the backend code base to be clear when there is conflict between locally stored metadata and the backend metadata. mainly the important bits.

aptalca · 2023-05-30T01:26:16Z

aptalca
May 30, 2023
Author

Unfortunately it's still not working for marking as unplayed on emby.

I started from scratch. Set up two local servers, one emby, one plex. I set up emby first and imported. Then set up plex and did a force export, and then enabled import/export. All good. Play state is perfectly in sync.

I mark a movie as watched on emby. Webhook shows up in log. item.markplayed shows up in db:list. Also shows up in db:queue.
I then do a state:push, it's pushed to plex. They're in sync again. All good.

I mark the same movie as unwatched in emby. Webhook shows up in docker log with a message saying it's getting queued. Nothing shows up in db:queue. db:list lists the item.markunplayed action, but its play state is still watched. yaml output of db:list shows it as watched on watchstate, watched on plex and unwatched on emby.

No matter what I do, it won't ever mark it as unwatched in watchstate and the servers are out of sync. I have to mark it as unplayed in plex, and then import and export. Then they get in sync again. I tried with a few different movies, all with the same result.

Weirdly, watchstate knows they are out of sync but doesn't do anything about it. When they're out of sync, where a movie is watched on plex but unwatched in emby, if I mark it as unwatched in plex, the next import shows the play state updating from plex. And the next export gladly shows No play state changes detected. because it was already marked as unplayed on emby.

Here's the yaml output after marking something as unplayed on emby and the webhook is received:

# docker exec -ti watchstate console db:list -l 1 -o yaml
-
  id: 3715
  type: movie
  updated: 2023-05-29T21:21:34-04:00
  watched: true
  via: plex
  title: Air
  year: 2023
  season: null
  episode: null
  parent: {  }
  guids:
    guid_imdb: tt16419074
    guid_tmdb: '964980'
    guid_tvdb: '343593'
  metadata:
    emby:
      id: '2065614'
      type: movie
      watched: '0'
      via: emby
      title: Air
      guids:
        guid_imdb: tt16419074
        guid_tmdb: '964980'
      added_at: '1683886581'
      extra:
        date: '2023-04-05'
      library: '54392'
      year: '2023'
      path: '/mnt/user/Movies/Air (2023)/Air (2023) WEBDL-720p ().mkv'
      played_at: '1683886581'
    plex:
      id: '253127'
      type: movie
      watched: '1'
      via: plex
      title: Air
      guids:
        guid_imdb: tt16419074
        guid_tmdb: '964980'
        guid_tvdb: '343593'
      added_at: '1683887118'
      extra:
        date: '2023-04-05'
      library: '8'
      year: '2023'
      path: '/tower/Movies/Air (2023)/Air (2023) WEBDL-720p ().mkv'
      played_at: '1685409694'
  extra:
    emby:
      event: item.markunplayed
      received_at: '2023-05-29T21:22:17-04:00'
    plex:
      event: task.import
      received_at: '2023-05-29T21:21:49-04:00'

4 replies

arabcoders May 30, 2023
Maintainer

Thank you. Once i have free time i will recheck emby code and hopefully fix it. as i am moving houses right now.

arabcoders Jun 3, 2023
Maintainer

@aptalca Hi, i pushed potential fix, please change your tag from arabcoders/watchstate:latest to arabcoders/watchstate:dev and try running it now.

aptalca Jun 3, 2023
Author

Yup, can confirm it's now working with the dev tag. Marked unplayed on emby, It correctly got marked in watchstate as unplayed, and pushed to other servers with a state:push. Thanks so much!!

arabcoders Jun 4, 2023
Maintainer

Good to know, i merged the changes to latest tag. you can switch back in an hour or so from me posting this.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Question regarding date comparison for watch state #317

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Replies: 6 comments 13 replies

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

Select a reply

Question regarding date comparison for watch state #317

aptalca May 26, 2023

Replies: 6 comments · 13 replies

arabcoders May 27, 2023 Maintainer

aptalca May 28, 2023 Author

arabcoders May 28, 2023 Maintainer

aptalca May 29, 2023 Author

arabcoders May 29, 2023 Maintainer

aptalca May 29, 2023 Author

arabcoders May 29, 2023 Maintainer

aptalca May 29, 2023 Author

arabcoders May 29, 2023 Maintainer

aptalca May 29, 2023 Author

arabcoders May 29, 2023 Maintainer

aptalca May 30, 2023 Author

arabcoders May 30, 2023 Maintainer

arabcoders Jun 3, 2023 Maintainer

aptalca Jun 3, 2023 Author

arabcoders Jun 4, 2023 Maintainer

aptalca
May 26, 2023

Replies: 6 comments 13 replies

arabcoders
May 27, 2023
Maintainer

aptalca
May 28, 2023
Author

arabcoders May 28, 2023
Maintainer

aptalca
May 29, 2023
Author

arabcoders
May 29, 2023
Maintainer

aptalca
May 29, 2023
Author

arabcoders May 29, 2023
Maintainer

aptalca May 29, 2023
Author

arabcoders May 29, 2023
Maintainer

aptalca May 29, 2023
Author

arabcoders May 29, 2023
Maintainer

aptalca
May 30, 2023
Author

arabcoders May 30, 2023
Maintainer

arabcoders Jun 3, 2023
Maintainer

aptalca Jun 3, 2023
Author

arabcoders Jun 4, 2023
Maintainer