[pysqlite] Concurrency

[Christian Boos]

Gerhard Häring wrote:
> ...
> You mean
> http://initd.org/tracker/pysqlite/attachment/ticket/126/reduced_timeout.patch
> ?
>   
Yes.
> I will try to understand the how and why - because at the moment I still
> cannot quite understand *why* it helps. I'll get back to you if I need a
> better explanation ...
In short, when you have a "writer contention" situation (i.e. what is 
described in #126),
you end up with two threads having a database lock situation.
At this point, the only way out is that one of the threads reaches its 
timeout,
allowing the other one to carry on.
So far, (with >= 2.0.5) we simply handle that situation correctly, i.e. 
the fact that
one of the threads hits the timeout actually enables the other one to 
carry on.

But for the succeeding thread, this had introduced a slow down equal to the
duration of the timeout.

Since the two threads are not exactly in a symmetrical situation,
one waiting for the EXCLUSIVE lock, the other for the RESERVED lock,
I thought it would be good to "help" the more advanced thread by giving it
more time to complete. That's why in the patch I introduce the 
`exclusive` flag
in `_sqlite_step_with_busyhandler`, and set it to 1 in places where I know
it will wait for the EXCLUSIVE lock. When that flag is 0, the timeout period
is reduced to a fifth of the original value.

Ok, I admit that it's a bit empirical, and *there could even be a better 
solution*:
if we would detect that we are in the "writer contention" situation, we 
should
immediately send a "Database Locked" exception to the thread that is not
waiting for the EXCLUSIVE lock. This won't change the outcome, simply
it will happen immediately, no need for a timeout at all.

-- Christian


*
*