在Golang中实施光标分页:Go Fiber,MySQL,从头开始
#go #pagination #fiber

介绍

分页是将大量记录或项目分为较小的块或页面的过程。分页很有用,因为它允许用户查看大量数据
以更易于管理的方式。

有几种不同的分页方法,包括编号的分页,其中显示了用户可以单击以在页面之间移动的页码和无限滚动,这些页码会在用户向下滚动页面时自动加载数据的新页面。光标分页是另一种方法,它使用光标或指针来跟踪用户在记录集中的位置,并允许他们使用光标在页面之间移动。

在本教程中,我们将使用scratch的GO,Fiber,Gorm和MySQL创建一个新的REST API。

先决条件

请确保系统上安装了以下依赖项

  • GO(需要1.16 /更高)< / li>
  • mysql 8

执行:

  • 让我们从创建一个新文件夹mkdir go-bookie开始。
  • 现在让我们发起一个新的GO mod:Go mod init“github.com/sadhakbj/go-bookie”
  • 现在我们需要安装GO纤维作为我们的依赖性。 pleae检查official documentation以获取有关GO纤维的所有详细信息
    • go get github.com/gofiber/fiber/v2
  • 我们还需要其他依赖项,因此也需要安装它们:
go get gorm.io/gorm
go get gorm.io/driver/mysql
go get github.com/google/uuid
  • 现在让我们创建新文件和文件夹:
touch main.go && mkdir models database helpers
  • 作为您的应用程序是Bookie,我们想列出书籍数据,因此让我们在文件夹模型中创建一个Model book.go
package models

import (
    "time"

    "github.com/google/uuid"
    "gorm.io/gorm"
)

type Book struct {
    ID          string `gorm:"primaryKey"`
    Title       string `json:"title" gorm:"type:varchar(191);not null"`
    Description string `json:"description" gorm:"type:text;not null"`
    Author      string `json:"author" gorm:"type:varchar(191);not null"`
    Price       uint   `json:"price" gorm:"not null"`
    CreatedAt   time.Time
    UpdatedAt   time.Time
}

func (book *Book) BeforeCreate(tx *gorm.DB) (err error) {
    book.ID = uuid.NewString()
    return
}

  • 我们将字段保持非常简单,我们将UUID用作主要键,使用Gorm,我们将使用BeforeCreate Hook
  • 将ID设置为新的UUID
  • 现在,让我们在文件夹数据库内创建一个新的文件数据库。
package database

import (
    "log"

    "github.com/sadhakbj/bookie-go/models"
    "gorm.io/driver/mysql"
    "gorm.io/gorm"
    "gorm.io/gorm/logger"
)

// DB is a global variable that represents the database connection
var DB *gorm.DB

// InitDB initializes the database connection
func InitDB() {
    // Connect to the database
    var err error
    dsn := "root:root@tcp(127.0.0.1:3306)/bookies?charset=utf8mb4&parseTime=True&loc=Local"
    db, err := gorm.Open(mysql.Open(dsn), &gorm.Config{})
    if err != nil {
        log.Fatal(err)
    }
    log.Println("Connected Successfully to Database")
    db.Logger = logger.Default.LogMode(logger.Info)
    log.Println("Running Migrations")
    db.AutoMigrate(&models.Book{})
    DB = db
}
  • 现在,让我们更新我们的主文件,即main.go可以连接到数据库,实例化新的光纤应用程序并将路由添加到它:
package main

import (
    "fmt"
    "log"
    "math/rand"
    "time"

    "github.com/gofiber/fiber/v2"
    "github.com/sadhakbj/bookie-go/controllers"
    "github.com/sadhakbj/bookie-go/database"
    "github.com/sadhakbj/bookie-go/models"
)

func main() {
    app := fiber.New()
    database.InitDB()

    app.Get("/books/seed", func(c *fiber.Ctx) error {
        var book models.Book
        if err := database.DB.Exec("delete from books where 1").Error; err != nil {
            return c.SendStatus(500)
        }
        for i := 1; i <= 20; i++ {
            book.Title = fmt.Sprintf("Book %d", i)
            book.Description = fmt.Sprintf("This is a description for a book %d", i)
            book.Price = uint(rand.Intn(500))
            book.Author = fmt.Sprintf("Book author %d", i)
            book.CreatedAt = time.Now().Add(-time.Duration(21-i) * time.Hour)

            database.DB.Create(&book)
        }

        return c.SendStatus(fiber.StatusOK)
    })

    app.Get("/books", controllers.GetPaginatedBooks)

    log.Fatal(app.Listen(":3000"))
}

我们在这里非常简单,我们使用新的HTTP端点来播种20个数据:/books/seed,我们还有另一个端点:'/books',它获取了分页的数据

  • 现在,我们在数据库中有20个数据,让我们继续在控制器文件夹中实现实际功能,所以让我们在文件夹controllers中创建一个新的bookscontroller.go
package controllers

import (
    "fmt"
    "strconv"

    "github.com/gofiber/fiber/v2"
    "github.com/sadhakbj/bookie-go/common"
    "github.com/sadhakbj/bookie-go/database"
    "github.com/sadhakbj/bookie-go/helpers"
    "github.com/sadhakbj/bookie-go/models"
)

func GetPaginatedBooks(c *fiber.Ctx) error {
    books := []models.Book{}
    perPage := c.Query("per_page", "10")
    sortOrder := c.Query("sort_order", "desc")
    cursor := c.Query("cursor", "")
    limit, err := strconv.ParseInt(perPage, 10, 64)
    if limit < 1 || limit > 100 {
        limit = 10
    }
    if err != nil {
        return c.Status(500).JSON("Invalid per_page option")
    }

    isFirstPage := cursor == ""
    pointsNext := false

    query := database.DB
    if cursor != "" {
        decodedCursor, err := helpers.DecodeCursor(cursor)
        if err != nil {
            fmt.Println(err)
            return c.SendStatus(500)
        }
        pointsNext = decodedCursor["points_next"] == true

        operator, order := getPaginationOperator(pointsNext, sortOrder)
        whereStr := fmt.Sprintf("(created_at %s ? OR (created_at = ? AND id %s ?))", operator, operator)
        query = query.Where(whereStr, decodedCursor["created_at"], decodedCursor["created_at"], decodedCursor["id"])
        if order != "" {
            sortOrder = order
        }
    }
    query.Order("created_at " + sortOrder).Limit(int(limit) + 1).Find(&books)
    hasPagination := len(books) > int(limit)

    if hasPagination {
        books = books[:limit]
    }

    if !isFirstPage && !pointsNext {
        books = helpers.Reverse(books)
    }

    pageInfo := calculatePagination(isFirstPage, hasPagination, int(limit), books, pointsNext)

    response := common.ResponseDTO{
        Success:    true,
        Data:       books,
        Pagination: pageInfo,
    }

    return c.Status(fiber.StatusOK).JSON(response)
}

func calculatePagination(isFirstPage bool, hasPagination bool, limit int, books []models.Book, pointsNext bool) helpers.PaginationInfo {
    pagination := helpers.PaginationInfo{}
    nextCur := helpers.Cursor{}
    prevCur := helpers.Cursor{}
    if isFirstPage {
        if hasPagination {
            nextCur := helpers.CreateCursor(books[limit-1].ID, books[limit-1].CreatedAt, true)
            pagination = helpers.GeneratePager(nextCur, nil)
        }
    } else {
        if pointsNext {
            // if pointing next, it always has prev but it might not have next
            if hasPagination {
                nextCur = helpers.CreateCursor(books[limit-1].ID, books[limit-1].CreatedAt, true)
            }
            prevCur = helpers.CreateCursor(books[0].ID, books[0].CreatedAt, false)
            pagination = helpers.GeneratePager(nextCur, prevCur)
        } else {
            // this is case of prev, there will always be nest, but prev needs to be calculated
            nextCur = helpers.CreateCursor(books[limit-1].ID, books[limit-1].CreatedAt, true)
            if hasPagination {
                prevCur = helpers.CreateCursor(books[0].ID, books[0].CreatedAt, false)
            }
            pagination = helpers.GeneratePager(nextCur, prevCur)
        }
    }
    return pagination
}

func getPaginationOperator(pointsNext bool, sortOrder string) (string, string) {
    if pointsNext && sortOrder == "asc" {
        return ">", ""
    }
    if pointsNext && sortOrder == "desc" {
        return "<", ""
    }
    if !pointsNext && sortOrder == "asc" {
        return "<", "desc"
    }
    if !pointsNext && sortOrder == "desc" {
        return ">", "asc"
    }

    return "", ""
}

在这里,主要方法:GetPaginatedBooks执行数据库查询以获取数据和方法:calculatePagination计算分页信息。计算分页的逻辑真的很简单,我们在数据库中执行查询(提供的限制+1),因此,如果结果计数>提供的限制限制,则意味着下一页 /上一页。< / p >

  • 您可以看到,我们缺少两个助手文件:common.gopagination.go,所以让我们也为它们编写内容。 (第一个文件旨在包括可以在整个应用程序上使用的通用辅助功能,而第二个则包括与分页相关的功能):
//common.go
package helpers

func Reverse[T any](s []T) []T {
    for i, j := 0, len(s)-1; i < j; i, j = i+1, j-1 {
        s[i], s[j] = s[j], s[i]
    }
    return s
}
package helpers

import (
    "encoding/base64"
    "encoding/json"
    "time"
)

type PaginationInfo struct {
    NextCursor string `json:"next_cursor"`
    PrevCursor string `json:"prev_cursor"`
}

type Cursor map[string]interface{}

func CreateCursor(id string, createdAt time.Time, pointsNext bool) Cursor {
    return Cursor{
        "id":          id,
        "created_at":  createdAt,
        "points_next": pointsNext,
    }
}

func GeneratePager(next Cursor, prev Cursor) PaginationInfo {
    return PaginationInfo{
        NextCursor: encodeCursor(next),
        PrevCursor: encodeCursor(prev),
    }
}

func encodeCursor(cursor Cursor) string {
    if len(cursor) == 0 {
        return ""
    }
    serializedCursor, err := json.Marshal(cursor)
    if err != nil {
        return ""
    }
    encodedCursor := base64.StdEncoding.EncodeToString(serializedCursor)
    return encodedCursor
}

func DecodeCursor(cursor string) (Cursor, error) {
    decodedCursor, err := base64.StdEncoding.DecodeString(cursor)
    if err != nil {
        return nil, err
    }

    var cur Cursor
    if err := json.Unmarshal(decodedCursor, &cur); err != nil {
        return nil, err
    }
    return cur, nil
}

  • 如果有分页,我们正在使用base64编码机制来编码光标值。它包括ID,创建的时间戳以及有关当前光标是否指向下一个的信息。光标是作为字符串生成的,并返回给用户。
  • 用户需要将光标作为查询参数发送,该参数已解码,并且数据库查询是相应执行的。
  • 最后,我们现在需要在一个名为response.go的文件夹中创建一个文件,该文件包括响应dto的结构:
package common

import "github.com/sadhakbj/bookie-go/helpers"

type ResponseDTO struct {
    Success    bool                   `json:"success"`
    Data       any                    `json:"data"`
    Pagination helpers.PaginationInfo `json:"pagination"`
}
  • 所以,是的,我们现在已经准备好实施了。现在,让我们运行服务器:go run main.go
  • 首先让我们播种我们的数据:http://localhost:3000/books/seed
  • 现在访问我们的HTTP端点:http://localhost:3000/books?per_page=2&sort_order=asc,它应该返回结果:
{
  "success": true,
  "data": [
    {
      "ID": "c55a8347-0b78-42f1-9b48-936c9c65361e",
      "title": "Book 1",
      "description": "This is a description for a book 1",
      "author": "Book author 1",
      "price": 81,
      "CreatedAt": "2022-12-17T16:14:47.952+09:00",
      "UpdatedAt": "2022-12-18T12:14:47.954+09:00"
    },
    {
      "ID": "6e4d9295-6b26-439c-928f-980b7361b662",
      "title": "Book 2",
      "description": "This is a description for a book 2",
      "author": "Book author 2",
      "price": 387,
      "CreatedAt": "2022-12-17T17:14:47.956+09:00",
      "UpdatedAt": "2022-12-18T12:14:47.954+09:00"
    }
  ],
  "pagination": {
    "next_cursor": "eyJjcmVhdGVkX2F0IjoiMjAyMi0xMi0xN1QxNzoxNDo0Ny45NTYrMDk6MDAiLCJpZCI6IjZlNGQ5Mjk1LTZiMjYtNDM5Yy05MjhmLTk4MGI3MzYxYjY2MiIsInBvaW50c19uZXh0Ijp0cnVlfQ==",
    "prev_cursor": ""
  }
}
  • 我们可以看到我们有分页信息并编码Next_cursor值,现在让我们尝试使用光标:http://localhost:3000/books?per_page=2&sort_order=asc&cursor=eyJjcmVhdGVkX2F0IjoiMjAyMi0xMi0xN1QxNzoxNDo0Ny45NTYrMDk6MDAiLCJpZCI6IjZlNGQ5Mjk1LTZiMjYtNDM5Yy05MjhmLTk4MGI3MzYxYjY2MiIsInBvaW50c19uZXh0Ijp0cnVlfQ==获取下一个2个项目:响应是:
{
  "success": true,
  "data": [
    {
      "ID": "62d843d7-889c-46c8-9956-12527b2b1a0d",
      "title": "Book 3",
      "description": "This is a description for a book 3",
      "author": "Book author 3",
      "price": 347,
      "CreatedAt": "2022-12-17T18:14:47.957+09:00",
      "UpdatedAt": "2022-12-18T12:14:47.954+09:00"
    },
    {
      "ID": "a0746960-a9da-4f62-be04-98e49ace03ef",
      "title": "Book 4",
      "description": "This is a description for a book 4",
      "author": "Book author 4",
      "price": 59,
      "CreatedAt": "2022-12-17T19:14:47.958+09:00",
      "UpdatedAt": "2022-12-18T12:14:47.954+09:00"
    }
  ],
  "pagination": {
    "next_cursor": "eyJjcmVhdGVkX2F0IjoiMjAyMi0xMi0xN1QxOToxNDo0Ny45NTgrMDk6MDAiLCJpZCI6ImEwNzQ2OTYwLWE5ZGEtNGY2Mi1iZTA0LTk4ZTQ5YWNlMDNlZiIsInBvaW50c19uZXh0Ijp0cnVlfQ==",
    "prev_cursor": "eyJjcmVhdGVkX2F0IjoiMjAyMi0xMi0xN1QxODoxNDo0Ny45NTcrMDk6MDAiLCJpZCI6IjYyZDg0M2Q3LTg4OWMtNDZjOC05OTU2LTEyNTI3YjJiMWEwZCIsInBvaW50c19uZXh0IjpmYWxzZX0="
  }
}

正如我们在这种情况下可以看到的,我们都有下一步和prev光标。

结论

我们很容易使用GO,GO ORM和MySQL实现光标Paginator。