optimize /inbox msg fetch query

There can be very slow queries when these conditions are met:
- two users have thousands of msgs in their convo
- a user has deleted all the messages
- some new messages were posted, less than 100 of them, say 50
- the user who deleted the thousands of messages loads the convo

What happens then is that lila fetches 100 messages for that user,
that have not been deleted by that user.
The query will find the latest 50 non-deleted messages,
then will look for more, and ends up scanning the thousands
of deleted messages. In vain: since messages can only be deleted
up to a point in time. There can't be non-deleted messages older
than a deleted message.

So, we removed the non-deleted condition from the query. We now
just fetch the last 100 messages (with possible skip for pagination),
deleted or not. And we filter them in lila after the fetch.

This guarantees that the DB will no longer scan thousands of
deleted messages.
pull/9879/head
Thibault Duplessis 2021-09-24 09:31:38 +02:00
parent 8961e18799
commit 5642398eb6
1 changed files with 7 additions and 2 deletions

View File

@ -214,14 +214,19 @@ final class MsgApi(
private def threadMsgsFor(threadId: MsgThread.Id, me: User, before: Option[DateTime]): Fu[List[Msg]] =
colls.msg
.find(
$doc("tid" -> threadId, "del" $ne me.id) ++ before.?? { b =>
$doc("tid" -> threadId) ++ before.?? { b =>
$doc("date" $lt b)
},
msgProjection
)
.sort($sort desc "date")
.cursor[Msg]()
.cursor[Bdoc]()
.list(msgsPerPage.value)
.map {
_.flatMap { doc =>
doc.getAsOpt[List[User.ID]]("del").fold(true)(!_.has(me.id)) ?? doc.asOpt[Msg]
}
}
private def setReadBy(threadId: MsgThread.Id, me: User, contactId: User.ID): Funit =
colls.thread.updateField(