LCOV - differential code coverage report
Current view: top level - src/include/backup - basebackup_sink.h (source / functions) Coverage Total Hit UBC CBC
Current: Differential Code Coverage 16@8cea358b128 vs 17@8cea358b128 Lines: 100.0 % 44 44 44
Current Date: 2024-04-14 14:21:10 Functions: 100.0 % 9 9 9
Baseline: 16@8cea358b128 Branches: 50.0 % 34 17 17 17
Baseline Date: 2024-04-14 14:21:09 Line coverage date bins:
Legend: Lines: hit not hit | Branches: + taken - not taken # not executed (240..) days: 100.0 % 44 44 44
Function coverage date bins:
(240..) days: 100.0 % 9 9 9
Branch coverage date bins:
(240..) days: 50.0 % 34 17 17 17

 Age         Owner                    Branch data    TLA  Line data    Source code
                                  1                 :                : /*-------------------------------------------------------------------------
                                  2                 :                :  *
                                  3                 :                :  * basebackup_sink.h
                                  4                 :                :  *    API for filtering or sending to a final destination the archives
                                  5                 :                :  *    produced by the base backup process
                                  6                 :                :  *
                                  7                 :                :  * Taking a base backup produces one archive per tablespace directory,
                                  8                 :                :  * plus a backup manifest unless that feature has been disabled. The
                                  9                 :                :  * goal of the backup process is to put those archives and that manifest
                                 10                 :                :  * someplace, possibly after postprocessing them in some way. A 'bbsink'
                                 11                 :                :  * is an object to which those archives, and the manifest if present,
                                 12                 :                :  * can be sent.
                                 13                 :                :  *
                                 14                 :                :  * In practice, there will be a chain of 'bbsink' objects rather than
                                 15                 :                :  * just one, with callbacks being forwarded from one to the next,
                                 16                 :                :  * possibly with modification. Each object is responsible for a
                                 17                 :                :  * single task e.g. command progress reporting, throttling, or
                                 18                 :                :  * communication with the client.
                                 19                 :                :  *
                                 20                 :                :  * Portions Copyright (c) 2010-2024, PostgreSQL Global Development Group
                                 21                 :                :  *
                                 22                 :                :  * src/include/backup/basebackup_sink.h
                                 23                 :                :  *
                                 24                 :                :  *-------------------------------------------------------------------------
                                 25                 :                :  */
                                 26                 :                : #ifndef BASEBACKUP_SINK_H
                                 27                 :                : #define BASEBACKUP_SINK_H
                                 28                 :                : 
                                 29                 :                : #include "access/xlogdefs.h"
                                 30                 :                : #include "common/compression.h"
                                 31                 :                : #include "nodes/pg_list.h"
                                 32                 :                : 
                                 33                 :                : /* Forward declarations. */
                                 34                 :                : struct bbsink;
                                 35                 :                : struct bbsink_ops;
                                 36                 :                : typedef struct bbsink bbsink;
                                 37                 :                : typedef struct bbsink_ops bbsink_ops;
                                 38                 :                : 
                                 39                 :                : /*
                                 40                 :                :  * Overall backup state shared by all bbsink objects for a backup.
                                 41                 :                :  *
                                 42                 :                :  * Before calling bbstate_begin_backup, caller must initiate a bbsink_state
                                 43                 :                :  * object which will last for the lifetime of the backup, and must thereafter
                                 44                 :                :  * update it as required before each new call to a bbsink method. The bbsink
                                 45                 :                :  * will retain a pointer to the state object and will consult it to understand
                                 46                 :                :  * the progress of the backup.
                                 47                 :                :  *
                                 48                 :                :  * 'tablespaces' is a list of tablespaceinfo objects. It must be set before
                                 49                 :                :  * calling bbstate_begin_backup() and must not be modified thereafter.
                                 50                 :                :  *
                                 51                 :                :  * 'tablespace_num' is the index of the current tablespace within the list
                                 52                 :                :  * stored in 'tablespaces'.
                                 53                 :                :  *
                                 54                 :                :  * 'bytes_done' is the number of bytes read so far from $PGDATA.
                                 55                 :                :  *
                                 56                 :                :  * 'bytes_total' is the total number of bytes estimated to be present in
                                 57                 :                :  * $PGDATA, if we have estimated this.
                                 58                 :                :  *
                                 59                 :                :  * 'bytes_total_is_valid' is true if and only if a proper estimate has been
                                 60                 :                :  * stored into 'bytes_total'.
                                 61                 :                :  *
                                 62                 :                :  * 'startptr' and 'starttli' identify the point in the WAL stream at which
                                 63                 :                :  * the backup began. They must be set before calling bbstate_begin_backup()
                                 64                 :                :  * and must not be modified thereafter.
                                 65                 :                :  */
                                 66                 :                : typedef struct bbsink_state
                                 67                 :                : {
                                 68                 :                :     List       *tablespaces;
                                 69                 :                :     int         tablespace_num;
                                 70                 :                :     uint64      bytes_done;
                                 71                 :                :     uint64      bytes_total;
                                 72                 :                :     bool        bytes_total_is_valid;
                                 73                 :                :     XLogRecPtr  startptr;
                                 74                 :                :     TimeLineID  starttli;
                                 75                 :                : } bbsink_state;
                                 76                 :                : 
                                 77                 :                : /*
                                 78                 :                :  * Common data for any type of basebackup sink.
                                 79                 :                :  *
                                 80                 :                :  * 'bbs_ops' is the relevant callback table.
                                 81                 :                :  *
                                 82                 :                :  * 'bbs_buffer' is the buffer into which data destined for the bbsink
                                 83                 :                :  * should be stored. It must be a multiple of BLCKSZ.
                                 84                 :                :  *
                                 85                 :                :  * 'bbs_buffer_length' is the allocated length of the buffer.
                                 86                 :                :  *
                                 87                 :                :  * 'bbs_next' is a pointer to another bbsink to which this bbsink is
                                 88                 :                :  * forwarding some or all operations.
                                 89                 :                :  *
                                 90                 :                :  * 'bbs_state' is a pointer to the bbsink_state object for this backup.
                                 91                 :                :  * Every bbsink associated with this backup should point to the same
                                 92                 :                :  * underlying state object.
                                 93                 :                :  *
                                 94                 :                :  * In general it is expected that the values of these fields are set when
                                 95                 :                :  * a bbsink is created and that they do not change thereafter. It's OK
                                 96                 :                :  * to modify the data to which bbs_buffer or bbs_state point, but no changes
                                 97                 :                :  * should be made to the contents of this struct.
                                 98                 :                :  */
                                 99                 :                : struct bbsink
                                100                 :                : {
                                101                 :                :     const bbsink_ops *bbs_ops;
                                102                 :                :     char       *bbs_buffer;
                                103                 :                :     size_t      bbs_buffer_length;
                                104                 :                :     bbsink     *bbs_next;
                                105                 :                :     bbsink_state *bbs_state;
                                106                 :                : };
                                107                 :                : 
                                108                 :                : /*
                                109                 :                :  * Callbacks for a base backup sink.
                                110                 :                :  *
                                111                 :                :  * All of these callbacks are required. If a particular callback just needs to
                                112                 :                :  * forward the call to sink->bbs_next, use bbsink_forward_<callback_name> as
                                113                 :                :  * the callback.
                                114                 :                :  *
                                115                 :                :  * Callers should always invoke these callbacks via the bbsink_* inline
                                116                 :                :  * functions rather than calling them directly.
                                117                 :                :  */
                                118                 :                : struct bbsink_ops
                                119                 :                : {
                                120                 :                :     /*
                                121                 :                :      * This callback is invoked just once, at the very start of the backup. It
                                122                 :                :      * must set bbs_buffer to point to a chunk of storage where at least
                                123                 :                :      * bbs_buffer_length bytes of data can be written.
                                124                 :                :      */
                                125                 :                :     void        (*begin_backup) (bbsink *sink);
                                126                 :                : 
                                127                 :                :     /*
                                128                 :                :      * For each archive transmitted to a bbsink, there will be one call to the
                                129                 :                :      * begin_archive() callback, some number of calls to the
                                130                 :                :      * archive_contents() callback, and then one call to the end_archive()
                                131                 :                :      * callback.
                                132                 :                :      *
                                133                 :                :      * Before invoking the archive_contents() callback, the caller should copy
                                134                 :                :      * a number of bytes equal to what will be passed as len into bbs_buffer,
                                135                 :                :      * but not more than bbs_buffer_length.
                                136                 :                :      *
                                137                 :                :      * It's generally good if the buffer is as full as possible before the
                                138                 :                :      * archive_contents() callback is invoked, but it's not worth expending
                                139                 :                :      * extra cycles to make sure it's absolutely 100% full.
                                140                 :                :      */
                                141                 :                :     void        (*begin_archive) (bbsink *sink, const char *archive_name);
                                142                 :                :     void        (*archive_contents) (bbsink *sink, size_t len);
                                143                 :                :     void        (*end_archive) (bbsink *sink);
                                144                 :                : 
                                145                 :                :     /*
                                146                 :                :      * If a backup manifest is to be transmitted to a bbsink, there will be
                                147                 :                :      * one call to the begin_manifest() callback, some number of calls to the
                                148                 :                :      * manifest_contents() callback, and then one call to the end_manifest()
                                149                 :                :      * callback. These calls will occur after all archives are transmitted.
                                150                 :                :      *
                                151                 :                :      * The rules for invoking the manifest_contents() callback are the same as
                                152                 :                :      * for the archive_contents() callback above.
                                153                 :                :      */
                                154                 :                :     void        (*begin_manifest) (bbsink *sink);
                                155                 :                :     void        (*manifest_contents) (bbsink *sink, size_t len);
                                156                 :                :     void        (*end_manifest) (bbsink *sink);
                                157                 :                : 
                                158                 :                :     /*
                                159                 :                :      * This callback is invoked just once, after all archives and the manifest
                                160                 :                :      * have been sent.
                                161                 :                :      */
                                162                 :                :     void        (*end_backup) (bbsink *sink, XLogRecPtr endptr, TimeLineID endtli);
                                163                 :                : 
                                164                 :                :     /*
                                165                 :                :      * If a backup is aborted by an error, this callback is invoked before the
                                166                 :                :      * bbsink object is destroyed, so that it can release any resources that
                                167                 :                :      * would not be released automatically. If no error occurs, this callback
                                168                 :                :      * is invoked after the end_backup callback.
                                169                 :                :      */
                                170                 :                :     void        (*cleanup) (bbsink *sink);
                                171                 :                : };
                                172                 :                : 
                                173                 :                : /* Begin a backup. */
                                174                 :                : static inline void
  891 rhaas@postgresql.org      175                 :CBC         316 : bbsink_begin_backup(bbsink *sink, bbsink_state *state, int buffer_length)
                                176                 :                : {
                                177         [ -  + ]:            316 :     Assert(sink != NULL);
                                178                 :                : 
                                179         [ -  + ]:            316 :     Assert(buffer_length > 0);
                                180                 :                : 
                                181                 :            316 :     sink->bbs_state = state;
                                182                 :            316 :     sink->bbs_buffer_length = buffer_length;
                                183                 :            316 :     sink->bbs_ops->begin_backup(sink);
                                184                 :                : 
                                185         [ -  + ]:            316 :     Assert(sink->bbs_buffer != NULL);
                                186         [ -  + ]:            316 :     Assert((sink->bbs_buffer_length % BLCKSZ) == 0);
                                187                 :            316 : }
                                188                 :                : 
                                189                 :                : /* Begin an archive. */
                                190                 :                : static inline void
                                191                 :            378 : bbsink_begin_archive(bbsink *sink, const char *archive_name)
                                192                 :                : {
                                193         [ -  + ]:            378 :     Assert(sink != NULL);
                                194                 :                : 
                                195                 :            378 :     sink->bbs_ops->begin_archive(sink, archive_name);
                                196                 :            378 : }
                                197                 :                : 
                                198                 :                : /* Process some of the contents of an archive. */
                                199                 :                : static inline void
                                200                 :         690430 : bbsink_archive_contents(bbsink *sink, size_t len)
                                201                 :                : {
                                202         [ -  + ]:         690430 :     Assert(sink != NULL);
                                203                 :                : 
                                204                 :                :     /*
                                205                 :                :      * The caller should make a reasonable attempt to fill the buffer before
                                206                 :                :      * calling this function, so it shouldn't be completely empty. Nor should
                                207                 :                :      * it be filled beyond capacity.
                                208                 :                :      */
                                209   [ +  -  -  + ]:         690430 :     Assert(len > 0 && len <= sink->bbs_buffer_length);
                                210                 :                : 
                                211                 :         690430 :     sink->bbs_ops->archive_contents(sink, len);
                                212                 :         690430 : }
                                213                 :                : 
                                214                 :                : /* Finish an archive. */
                                215                 :                : static inline void
                                216                 :            365 : bbsink_end_archive(bbsink *sink)
                                217                 :                : {
                                218         [ -  + ]:            365 :     Assert(sink != NULL);
                                219                 :                : 
                                220                 :            365 :     sink->bbs_ops->end_archive(sink);
                                221                 :            365 : }
                                222                 :                : 
                                223                 :                : /* Begin the backup manifest. */
                                224                 :                : static inline void
                                225                 :            301 : bbsink_begin_manifest(bbsink *sink)
                                226                 :                : {
                                227         [ -  + ]:            301 :     Assert(sink != NULL);
                                228                 :                : 
                                229                 :            301 :     sink->bbs_ops->begin_manifest(sink);
                                230                 :            301 : }
                                231                 :                : 
                                232                 :                : /* Process some of the manifest contents. */
                                233                 :                : static inline void
                                234                 :           1547 : bbsink_manifest_contents(bbsink *sink, size_t len)
                                235                 :                : {
                                236         [ -  + ]:           1547 :     Assert(sink != NULL);
                                237                 :                : 
                                238                 :                :     /* See comments in bbsink_archive_contents. */
                                239   [ +  -  -  + ]:           1547 :     Assert(len > 0 && len <= sink->bbs_buffer_length);
                                240                 :                : 
                                241                 :           1547 :     sink->bbs_ops->manifest_contents(sink, len);
                                242                 :           1547 : }
                                243                 :                : 
                                244                 :                : /* Finish the backup manifest. */
                                245                 :                : static inline void
                                246                 :            301 : bbsink_end_manifest(bbsink *sink)
                                247                 :                : {
                                248         [ -  + ]:            301 :     Assert(sink != NULL);
                                249                 :                : 
                                250                 :            301 :     sink->bbs_ops->end_manifest(sink);
                                251                 :            301 : }
                                252                 :                : 
                                253                 :                : /* Finish a backup. */
                                254                 :                : static inline void
                                255                 :            303 : bbsink_end_backup(bbsink *sink, XLogRecPtr endptr, TimeLineID endtli)
                                256                 :                : {
                                257         [ -  + ]:            303 :     Assert(sink != NULL);
                                258         [ -  + ]:            303 :     Assert(sink->bbs_state->tablespace_num == list_length(sink->bbs_state->tablespaces));
                                259                 :                : 
                                260                 :            303 :     sink->bbs_ops->end_backup(sink, endptr, endtli);
                                261                 :            303 : }
                                262                 :                : 
                                263                 :                : /* Release resources before destruction. */
                                264                 :                : static inline void
                                265                 :            296 : bbsink_cleanup(bbsink *sink)
                                266                 :                : {
                                267         [ -  + ]:            296 :     Assert(sink != NULL);
                                268                 :                : 
                                269                 :            296 :     sink->bbs_ops->cleanup(sink);
                                270                 :            296 : }
                                271                 :                : 
                                272                 :                : /* Forwarding callbacks. Use these to pass operations through to next sink. */
                                273                 :                : extern void bbsink_forward_begin_backup(bbsink *sink);
                                274                 :                : extern void bbsink_forward_begin_archive(bbsink *sink,
                                275                 :                :                                          const char *archive_name);
                                276                 :                : extern void bbsink_forward_archive_contents(bbsink *sink, size_t len);
                                277                 :                : extern void bbsink_forward_end_archive(bbsink *sink);
                                278                 :                : extern void bbsink_forward_begin_manifest(bbsink *sink);
                                279                 :                : extern void bbsink_forward_manifest_contents(bbsink *sink, size_t len);
                                280                 :                : extern void bbsink_forward_end_manifest(bbsink *sink);
                                281                 :                : extern void bbsink_forward_end_backup(bbsink *sink, XLogRecPtr endptr,
                                282                 :                :                                       TimeLineID endtli);
                                283                 :                : extern void bbsink_forward_cleanup(bbsink *sink);
                                284                 :                : 
                                285                 :                : /* Constructors for various types of sinks. */
                                286                 :                : extern bbsink *bbsink_copystream_new(bool send_to_client);
                                287                 :                : extern bbsink *bbsink_gzip_new(bbsink *next, pg_compress_specification *);
                                288                 :                : extern bbsink *bbsink_lz4_new(bbsink *next, pg_compress_specification *);
                                289                 :                : extern bbsink *bbsink_zstd_new(bbsink *next, pg_compress_specification *);
                                290                 :                : extern bbsink *bbsink_progress_new(bbsink *next, bool estimate_backup_size);
                                291                 :                : extern bbsink *bbsink_server_new(bbsink *next, char *pathname);
                                292                 :                : extern bbsink *bbsink_throttle_new(bbsink *next, uint32 maxrate);
                                293                 :                : 
                                294                 :                : /* Extra interface functions for progress reporting. */
                                295                 :                : extern void basebackup_progress_wait_checkpoint(void);
                                296                 :                : extern void basebackup_progress_estimate_backup_size(void);
                                297                 :                : extern void basebackup_progress_wait_wal_archive(bbsink_state *);
                                298                 :                : extern void basebackup_progress_transfer_wal(void);
                                299                 :                : extern void basebackup_progress_done(void);
                                300                 :                : 
                                301                 :                : #endif
        

Generated by: LCOV version 2.1-beta2-3-g6141622