connectors/ConnectorBase.py
author Edouard Tisserant
Wed, 05 Dec 2018 13:03:09 +0100
changeset 2465 47d3aea2be30
parent 2463 8742337a9fe3
child 2487 6a4f9a061994
permissions -rw-r--r--
Bigger chunks, from 16k to 1M
2463
8742337a9fe3 Chunk based transfer for PLC binary and extra files, and some collateral code refactoring.
Edouard Tisserant
parents:
diff changeset
     1
#!/usr/bin/env python
8742337a9fe3 Chunk based transfer for PLC binary and extra files, and some collateral code refactoring.
Edouard Tisserant
parents:
diff changeset
     2
# -*- coding: utf-8 -*-
8742337a9fe3 Chunk based transfer for PLC binary and extra files, and some collateral code refactoring.
Edouard Tisserant
parents:
diff changeset
     3
8742337a9fe3 Chunk based transfer for PLC binary and extra files, and some collateral code refactoring.
Edouard Tisserant
parents:
diff changeset
     4
# See COPYING file for copyrights details.
8742337a9fe3 Chunk based transfer for PLC binary and extra files, and some collateral code refactoring.
Edouard Tisserant
parents:
diff changeset
     5
8742337a9fe3 Chunk based transfer for PLC binary and extra files, and some collateral code refactoring.
Edouard Tisserant
parents:
diff changeset
     6
import md5
8742337a9fe3 Chunk based transfer for PLC binary and extra files, and some collateral code refactoring.
Edouard Tisserant
parents:
diff changeset
     7
8742337a9fe3 Chunk based transfer for PLC binary and extra files, and some collateral code refactoring.
Edouard Tisserant
parents:
diff changeset
     8
class ConnectorBase(object):
8742337a9fe3 Chunk based transfer for PLC binary and extra files, and some collateral code refactoring.
Edouard Tisserant
parents:
diff changeset
     9
2465
47d3aea2be30 Bigger chunks, from 16k to 1M
Edouard Tisserant
parents: 2463
diff changeset
    10
    #chuncksize = 16384
47d3aea2be30 Bigger chunks, from 16k to 1M
Edouard Tisserant
parents: 2463
diff changeset
    11
    chuncksize = 1024*1024
2463
8742337a9fe3 Chunk based transfer for PLC binary and extra files, and some collateral code refactoring.
Edouard Tisserant
parents:
diff changeset
    12
    def BlobFromFile(self, filepath): 
8742337a9fe3 Chunk based transfer for PLC binary and extra files, and some collateral code refactoring.
Edouard Tisserant
parents:
diff changeset
    13
        s = md5.new()
8742337a9fe3 Chunk based transfer for PLC binary and extra files, and some collateral code refactoring.
Edouard Tisserant
parents:
diff changeset
    14
        blobID = s.digest()  # empty md5, to support empty blob
8742337a9fe3 Chunk based transfer for PLC binary and extra files, and some collateral code refactoring.
Edouard Tisserant
parents:
diff changeset
    15
        with open(filepath, "rb") as f:
8742337a9fe3 Chunk based transfer for PLC binary and extra files, and some collateral code refactoring.
Edouard Tisserant
parents:
diff changeset
    16
            while True:
8742337a9fe3 Chunk based transfer for PLC binary and extra files, and some collateral code refactoring.
Edouard Tisserant
parents:
diff changeset
    17
                chunk = f.read(self.chuncksize) 
8742337a9fe3 Chunk based transfer for PLC binary and extra files, and some collateral code refactoring.
Edouard Tisserant
parents:
diff changeset
    18
                if len(chunk) == 0: return blobID
8742337a9fe3 Chunk based transfer for PLC binary and extra files, and some collateral code refactoring.
Edouard Tisserant
parents:
diff changeset
    19
                blobID = self.AppendChunkToBlob(chunk, blobID)
8742337a9fe3 Chunk based transfer for PLC binary and extra files, and some collateral code refactoring.
Edouard Tisserant
parents:
diff changeset
    20
                s.update(chunk)
8742337a9fe3 Chunk based transfer for PLC binary and extra files, and some collateral code refactoring.
Edouard Tisserant
parents:
diff changeset
    21
                if blobID != s.digest(): return None
8742337a9fe3 Chunk based transfer for PLC binary and extra files, and some collateral code refactoring.
Edouard Tisserant
parents:
diff changeset
    22