Django Mixin to Find Any Object's Page Number in Paginated QuerySets

Background

This article explores creating a reusable Python class mixin that provides Django class-based views methods for determining an object's page number within a paginated QuerySet. The problem this mixin looks to solve is a scenario where you leave a paginated ListView to perform an operation on a particular object. Upon completion, you want to return to the page the object belongs to rather than start again at the beginning of the QuerySet (e.g. the first page). This mixin accomplishes exactly this and abstracts the logic so it can be flexibly applied in different situations.

My earlier article Find An Object's Page Number Within A Django Paginated QuerySet establishes the core logic of this mixin. If you're unsure of the intent behind any of the code and it's not explained in detail here, take a look at this resource.

Improvements

This mixin improves upon the logic implemented in the article Find An Object's Page Number Within A Django Paginated QuerySet, where the entirety of the code was confined to the get_success_url method. Much of where this mixin differs was introduced in the article's "Other Considerations" section as suggestions for improvement. Notably, the mixin:

• Handles large datasets gracefully by using the QuerySet.iterator() method to chunk large queries instead of loading everything into memory, and
• Provides the option to exclude fields from the query that contain large data (consider a field that stores a blog model's content).

Beyond what was touched on in the earlier article, the mixin:

• Caches the generated page number to avoid redundant computation,
• Constructs the ordering criteria using the QuerySet rather than the view class in case a view utilizes custom managers or default ordering that wouldn't be returned by the ListView's get_ordering method, and
• Handles DeleteView where an object's page may no longer exist if it's the last object in a QuerySet and the first object on a page.

The Code

# mixins.py

import logging

from django.views.generic import ListView
from django.views.generic.edit import DeleteView
from django.db.models.expressions import Window
from django.db.models.functions import RowNumber
from django.db.models import F
# from django.db import connection, reset_queries


logger = logging.getLogger(__name__)


class PageNumberMixin:
    """
    Mixin that calculates which page an object appears on in a paginated ListView.

    Requires:
    - list_view_class: The ListView class to use for ordering/pagination settings
    - self.object: The object to find the page for
    """

    list_view_class = None

    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        self.page_number = None

    def get_page_number(
            self,
            qset=None,
            adjust_for_delete=True,
            exclude_fields=None,
    ):
        # reset_queries()
        if self.page_number:
            return self.page_number

        self.page_number = 1

        # check list_view_class attribute is properly set
        if self.list_view_class is None:
            raise AttributeError(f'Attribute "list_view_class" must be set.')
        try:
            if not issubclass(self.list_view_class, ListView):
                raise AttributeError(f'Attribute "list_view_class" must '
                                     f'inherit from ListView, got '
                                     f'{self.list_view_class.__name__} '
                                     f'instead.')
        except TypeError:
            raise AttributeError(
                f'Attribute "list_view_class" must be a class that inherits '
                f'from ListView, got {type(self.list_view_class).__name__}.')

        list_view = self.list_view_class()

        paginate_by = list_view.paginate_by

        # noinspection PyUnresolvedReferences
        if not paginate_by or self.object is None:
            # can't determine page
            logger.warning(f'Unable to determine page number due to one or '
                           f'more required attributes unset.')
            return self.page_number

        # if qset wasn't provided, get it from the view
        if not qset:
            qset = list_view.get_queryset()

        ordering = qset.query.order_by
        if not ordering:
            # get fallback ordering
            ordering = qset.model._meta.ordering

        if isinstance(ordering, str):
            ordering = (ordering,)

        expressions = []
        for field in ordering:
            if field.startswith('-'):
                # descending
                expressions.append(F(field[1:]).desc())
            else:
                expressions.append(F(field))

        # annotate the qset with page numbers
        qset = qset.annotate(
            row_number=Window(
                expression=RowNumber(), order_by=expressions
            )
        )

        # defer fields, if any
        if exclude_fields:
            if isinstance(exclude_fields, str):
                exclude_fields = (exclude_fields,)
            qset = qset.defer(*exclude_fields)

        # evaluate the query and determine the object's row number
        tgt_row = None
        for item in qset.iterator():
            # noinspection PyUnresolvedReferences
            if item.pk == self.object.pk:
                # use n - 1 for deletes on form submission
                # otherwise, you might find yourself on a page that doesn't
                #   exist
                # if not a form submission, you want to return to actual page
                if (issubclass(self.__class__, DeleteView)
                        and adjust_for_delete):
                    tgt_row = max(item.row_number - 1, 1)
                else:
                    tgt_row = item.row_number
                break
        # print(len(connection.queries))
        if tgt_row:
            self.page_number = ((tgt_row - 1) // paginate_by) + 1

        return self.page_number

    def get_url_with_page_number(self, base_url=None):
        if base_url is None:
            # noinspection PyUnresolvedReferences
            base_url = self.get_success_url()

        page_num = self.get_page_number()
        if page_num > 1:
            return '%s?page=%s' % (base_url, page_num)
        return base_url

What's Happening

The mixin has a single class-level attribute list_view_class that stores the ListView class we're looking to return to. We'll instantiate an instance of this class and use it to access attributes and methods like paginate_by and get_queryset. Additionally we have an instance-level attribute self.page_number which we'll use to cache the results of our get_page_number method to reduce redundant operations and unnecessary database hits. Last, we have two methods get_page_number and get_url_with_page_number. The former is where all the magic happens and the latter is an implementation of the former in its most likely use case.

Get Page Number

The first method containing the majority of our logic is get_page_number. This method accepts arguments for "qset," "adjust_for_delete," and "exclude_fields" with all being optional. Really the only requirement for setup is for the class-level attribute list_view_class to be provided. Everything else is determined dynamically.

First, the method checks if there's a cached value.

        if self.page_number:
            return self.page_number

        self.page_number = 1

If a cached value is present, this means the method has already run at least once. Rather than repeat operations that will yield the same result, the cached value is returned and we exit the method. If a cached value isn't found, we set the page_number instance attribute to a default value of 1.

Next, we run a couple checks on the view class provided by the class-level attribute list_view_class.

        # check list_view_class attribute is properly set
        if self.list_view_class is None:
            raise AttributeError(f'Attribute "list_view_class" must be set.')
        try:
            if not issubclass(self.list_view_class, ListView):
                raise AttributeError(f'Attribute "list_view_class" must '
                                     f'inherit from ListView, got '
                                     f'{self.list_view_class.__name__} '
                                     f'instead.')
        except TypeError:
            raise AttributeError(
                f'Attribute "list_view_class" must be a class that inherits '
                f'from ListView, got {type(self.list_view_class).__name__}.')

        list_view = self.list_view_class()

Because much of the logic depends on what obtain from this attribute, it's critical this is properly set. If something looks amiss, exceptions are raised with descriptive errors to help debug the problem. If everything looks good, we instantiate an instance of the class.

With a class instance available, we now retrieve paginate_by which tells us how many objects are displayed on each page. A quick check is run to make sure this value is set. There's no point to further operations if not.

        if not paginate_by or self.object is None:
            # can't determine page
            logger.warning(f'Unable to determine page number due to one or '
                           f'more required attributes unset.')
            return self.page_number

Next, we retrieve our QuerySet and determine ordering.

        if not qset:
            qset = list_view.get_queryset()

        ordering = qset.query.order_by
        if not ordering:
            # get fallback ordering
            ordering = qset.model._meta.ordering

        if isinstance(ordering, str):
            ordering = (ordering,)

Whereas previously I obtained the ordering from the ListView's get_ordering() method, here I obtain the ordering directly from the query and then from the model as a fallback. This way, no matter how the QuerySet is provided, we have a good chance of my ordering matching what's present in the list we're returning to.

Next, we construct the expressions and annotate the QuerySet with row numbers. This is unchanged from the earlier article Find An Object's Page Number Within A Django Paginated QuerySet.

        expressions = []
        for field in ordering:
            if field.startswith('-'):
                # descending
                expressions.append(F(field[1:]).desc())
            else:
                expressions.append(F(field))

        # annotate the qset with page numbers
        qset = qset.annotate(
            row_number=Window(
                expression=RowNumber(), order_by=expressions
            )
        )

Next, we update our query to exclude any fields we desire. The field(s) are passed via the "exclude_fields" argument in either string or tuple form. If present, we use the QuerySet method defer() to exclude those fields from the query. Again, this helps us build lighter queries if our table has large fields that may consume a lot of memory.

Last, we search our resulting QuerySet for the record that matches our object's ID and determine the page number the object should exist at.

        # evaluate the query and determine the object's row number
        tgt_row = None
        for item in qset.iterator():
            # noinspection PyUnresolvedReferences
            if item.pk == self.object.pk:
                # use n - 1 for deletes on form submission
                # otherwise, you might find yourself on a page that doesn't
                #   exist
                # if not a form submission, you want to return to actual page
                if (issubclass(self.__class__, DeleteView)
                        and adjust_for_delete):
                    tgt_row = max(item.row_number - 1, 1)
                else:
                    tgt_row = item.row_number
                break
        # print(len(connection.queries))
        if tgt_row:
            self.page_number = ((tgt_row - 1) // paginate_by) + 1

        return self.page_number

This differs from the earlier article in its handling of DeleteView. As mentioned before, if we have an object that's last in the set but first on the last page, the page will no longer exist following the object's deletion. Therefore we actually want the row number of the object just before for determining the appropriate page number.

But a deletion may not always be what occurs, consider dismiss buttons like "cancel" or "back." These actions bypass deletion. In this case, we do want to return to the object's page since it will still exist. I handle this with the boolean "adjust_for_delete" argument and defaults to True. If you're calculating the actual object's page from a DeleteView, you should set this argument to False.

And that's that. We have a method that determines where an object lies within a paginated QuerySet.

Get URL With Page Number

We also have a method that constructs a URL with a query parameter for the computed page number. I added this method to the mixin because I mostly intend to use the mixin to build URLs anyway so might as well abstract that logic instead of repeating code.

    def get_url_with_page_number(self, base_url=None):
        if base_url is None:
            # noinspection PyUnresolvedReferences
            base_url = self.get_success_url()

        page_num = self.get_page_number()
        if page_num > 1:
            return '%s?page=%s' % (base_url, page_num)
        return base_url

The method accepts an optional argument "base_url" for use with constructing the complete URL. If not provided, the view's get_success_url() method is utilized as a fall back. Even when using get_success_url(), I still tend to explicitly call the method before and pass it as an argument.

    def get_success_url(self):
        success_url = super().get_success_url()
        return self.get_url_with_page_number(success_url)

Lastly and as with before, I exclude pagination from page one for stylistic reasons (it would still work if I did but what's the point?).

Usage In The Wild

Here's how this mixin can be used with an UpdateView assuming it's located in a sibling module "mixins.py."

# views.py

from django.views.generic import ListView, UpdateView
from django.urls import reverse_lazy

from .models import MyModel
from .mixins import PageNumberMixin


class MyListView(ListView):
    ...


class MyUpdateView(PageNumberMixin, UpdateView):
    ...
    list_view_class = MyListView

    def get_context_data(self, **kwargs):
        context = super().get_context_data(**kwargs)
        context['dismiss_url'] = self.get_success_url()
        return context

    def get_success_url(self):
        success_url = super().get_success_url()
        return self.get_url_with_page_number(success_url)

In get_context_data, I add a "dismiss_url" option to use with cancel/dismiss buttons within the template.

Final Thoughts

In this article we looked at how to construct a mixin to solve the common problem of finding the page you were on before leaving to perform an operation. The methods contained provide a stable and efficient solution to this problem and are well adapted to be used in different and varied situations. If you haven't already, take a look at the article Find An Object's Page Number Within A Django Paginated QuerySet which looks more closely at the query itself and how it's constructed. By abstracting this logic into a reusable mixin, we've transformed a complex, one-off solution into a maintainable tool that can enhance user experience across your entire Django application.