まあこれまでのMastodonと比べればbreaking changeなのはこっちの方なので、これはこれで慣れてたじゃんというのは、まあ、正しい…

@unarist What's this about? 🤔

@Gargron discussing about https://github.com/tootsuite/mastodon/pull/5211#issuecomment-334105958 with clworld

@unarist Hmmm. I don't think this breaks since_id. Even if it's newly-fetched status, since_id is about chronology. "Give me toots written after this"

I agree with the concern of faking time.

The delay should not be a problem since regardless of when sidekiq executes delivery, "published" is the original created_at?

@Gargron New behavior seems correct for published order, but it's different from current one, arrived order. We can fetch complete timeline by since_id={cached_statuses.latest.id} now, but it's not guaranteed with the new behavior.

Technically, 1 msec delay of status arriving may cause losing it from timeline for apps which only use REST API (e.g. Amaroq). Also ISO8601 encoding strips msec part, so it will be past timestamp even if zero latency on delivery. The case is not rare.

cc @Clworld

@unarist @Clworld But arrived order was wrong all along... If you refresh someone's profile with since_id, do you expect a 3mo old status to show up in the results? Even if it just arrived?

@unarist @Gargron I think really big delay isn't caused by Sidekiq delay and it's caused by other functions (fetching boosted original status, search toot url, etc). I suggest arrival numbering for simple toot delivery.(Sidekiq delay is not big because it will retry only 3 times(OStatus) and 8 times(ActivityPub))

@Clworld @unarist Aha, I think I can understand your concern now. It's the very short term delay fluctuations that could mess up since_id. Yes, I understand. Okay. Then there needs to be a flag on the processing classes that's true only when they're called from the inbox controller.

@unarist @Clworld So when called from inbox controller -> local timestamp; otherwise -> created_at timestamp

@Clworld @unarist And if created_at timestamp is in the future, fallback to local timestamp too.

@unarist @Gargron Yes, that's what I suggest! thx!

@Clworld @Gargron btw, since chronological order of lower bits of new id is not guaranteed, new status may have past id if both statuses were created in a millisecond, I think.

If so, using latest status id as since_id is still may cause fetching leakage, which many apps (incl. WebUI) does and we've provided via Link header https://github.com/tootsuite/mastodon/blob/d5091387c6ddbe03b118b0cfb6d74cf821b84fb2/app/controllers/api/v1/timelines/public_controller.rb#L48 If client wants to new statuses without leakage, they should clear lower bits before use for since_id.

@unarist @Gargron Note: Remember that Home TL ordering redis keys are treated as double and only have 53bit precision and msec timestamp is already 41 bit length and Snowflake id will be 57bit now. So, last 4 bit on Snowflake id is ignored on redis Home TL sorting key.