connectors/ConnectorBase.py
author GP Orcullo <kinsamanka@gmail.com>
Tue, 01 Nov 2022 14:21:16 +0800
branchpython3
changeset 3772 ec2babbd5698
parent 3750 f62625418bff
child 3808 3e219f00151a
permissions -rw-r--r--
str encoding fixes
2463
8742337a9fe3 Chunk based transfer for PLC binary and extra files, and some collateral code refactoring.
Edouard Tisserant
parents:
diff changeset
     1
#!/usr/bin/env python
8742337a9fe3 Chunk based transfer for PLC binary and extra files, and some collateral code refactoring.
Edouard Tisserant
parents:
diff changeset
     2
# -*- coding: utf-8 -*-
8742337a9fe3 Chunk based transfer for PLC binary and extra files, and some collateral code refactoring.
Edouard Tisserant
parents:
diff changeset
     3
8742337a9fe3 Chunk based transfer for PLC binary and extra files, and some collateral code refactoring.
Edouard Tisserant
parents:
diff changeset
     4
# See COPYING file for copyrights details.
8742337a9fe3 Chunk based transfer for PLC binary and extra files, and some collateral code refactoring.
Edouard Tisserant
parents:
diff changeset
     5
3750
f62625418bff automated conversion using 2to3-3.9 tool
GP Orcullo <kinsamanka@gmail.com>
parents: 2621
diff changeset
     6
2540
fca79ca84272 Replace md5 module with hashlib
Andrey Skvortsov <andrej.skvortzov@gmail.com>
parents: 2492
diff changeset
     7
import hashlib
2463
8742337a9fe3 Chunk based transfer for PLC binary and extra files, and some collateral code refactoring.
Edouard Tisserant
parents:
diff changeset
     8
2492
7dd551ac2fa0 check_sources.sh makes me become even less productive
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 2487
diff changeset
     9
2463
8742337a9fe3 Chunk based transfer for PLC binary and extra files, and some collateral code refactoring.
Edouard Tisserant
parents:
diff changeset
    10
class ConnectorBase(object):
8742337a9fe3 Chunk based transfer for PLC binary and extra files, and some collateral code refactoring.
Edouard Tisserant
parents:
diff changeset
    11
2465
47d3aea2be30 Bigger chunks, from 16k to 1M
Edouard Tisserant
parents: 2463
diff changeset
    12
    chuncksize = 1024*1024
2492
7dd551ac2fa0 check_sources.sh makes me become even less productive
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 2487
diff changeset
    13
2487
6a4f9a061994 Reworked chunk based transfer to support duplicated files (i.e. files with same content, but different names)
Edouard Tisserant
parents: 2465
diff changeset
    14
    def BlobFromFile(self, filepath, seed):
2540
fca79ca84272 Replace md5 module with hashlib
Andrey Skvortsov <andrej.skvortzov@gmail.com>
parents: 2492
diff changeset
    15
        s = hashlib.new('md5')
3772
ec2babbd5698 str encoding fixes
GP Orcullo <kinsamanka@gmail.com>
parents: 3750
diff changeset
    16
        s.update(seed.encode())
2487
6a4f9a061994 Reworked chunk based transfer to support duplicated files (i.e. files with same content, but different names)
Edouard Tisserant
parents: 2465
diff changeset
    17
        blobID = self.SeedBlob(seed)
2463
8742337a9fe3 Chunk based transfer for PLC binary and extra files, and some collateral code refactoring.
Edouard Tisserant
parents:
diff changeset
    18
        with open(filepath, "rb") as f:
2487
6a4f9a061994 Reworked chunk based transfer to support duplicated files (i.e. files with same content, but different names)
Edouard Tisserant
parents: 2465
diff changeset
    19
            while blobID == s.digest():
2492
7dd551ac2fa0 check_sources.sh makes me become even less productive
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 2487
diff changeset
    20
                chunk = f.read(self.chuncksize)
7dd551ac2fa0 check_sources.sh makes me become even less productive
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 2487
diff changeset
    21
                if len(chunk) == 0:
7dd551ac2fa0 check_sources.sh makes me become even less productive
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 2487
diff changeset
    22
                    return blobID
2463
8742337a9fe3 Chunk based transfer for PLC binary and extra files, and some collateral code refactoring.
Edouard Tisserant
parents:
diff changeset
    23
                blobID = self.AppendChunkToBlob(chunk, blobID)
8742337a9fe3 Chunk based transfer for PLC binary and extra files, and some collateral code refactoring.
Edouard Tisserant
parents:
diff changeset
    24
                s.update(chunk)
2621
af09744a468e Better error handling when blob transfer fail
Edouard Tisserant
parents: 2542
diff changeset
    25
        raise IOError("Data corrupted during transfer or connection lost")