author | Edouard Tisserant |
Mon, 14 Jun 2021 16:48:39 +0200 | |
changeset 2741 | 3cc5663af196 |
child 2742 | 5f7445b582d4 |
permissions | -rw-r--r-- |
2741
3cc5663af196
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
1 |
#!/usr/bin/env python |
3cc5663af196
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
2 |
# -*- coding: utf-8 -*- |
3cc5663af196
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
3 |
|
3cc5663af196
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
4 |
# This file is part of Beremiz |
3cc5663af196
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
5 |
# Copyright (C) 2021: Edouard TISSERANT |
3cc5663af196
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
6 |
# |
3cc5663af196
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
7 |
# See COPYING file for copyrights details. |
3cc5663af196
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
8 |
|
3cc5663af196
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
9 |
# Based on Eelco Hoogendoorn stackoverflow answer about RingBuffer with numpy |
3cc5663af196
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
10 |
|
3cc5663af196
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
11 |
import numpy as np |
3cc5663af196
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
12 |
|
3cc5663af196
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
13 |
|
3cc5663af196
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
14 |
class RingBuffer(object): |
3cc5663af196
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
15 |
def __init__(self, width=None, size=65536, padding=None): |
3cc5663af196
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
16 |
self.size = size |
3cc5663af196
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
17 |
self.padding = size if padding is None else padding |
3cc5663af196
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
18 |
shape = (self.size+self.padding,) |
3cc5663af196
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
19 |
if width : |
3cc5663af196
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
20 |
shape += (width,) |
3cc5663af196
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
21 |
self.buffer = np.zeros(shape) |
3cc5663af196
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
22 |
self.counter = 0 |
3cc5663af196
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
23 |
self.full = False |
3cc5663af196
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
24 |
|
3cc5663af196
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
25 |
def append(self, data): |
3cc5663af196
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
26 |
"""this is an O(n) operation""" |
3cc5663af196
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
27 |
data = data[-self.padding:] |
3cc5663af196
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
28 |
n = len(data) |
3cc5663af196
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
29 |
if self.remaining < n: self.compact() |
3cc5663af196
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
30 |
self.buffer[self.counter+self.size:][:n] = data |
3cc5663af196
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
31 |
self.counter += n |
3cc5663af196
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
32 |
|
3cc5663af196
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
33 |
@property |
3cc5663af196
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
34 |
def count(self): |
3cc5663af196
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
35 |
return self.counter if not self.full else self.size |
3cc5663af196
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
36 |
|
3cc5663af196
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
37 |
@property |
3cc5663af196
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
38 |
def remaining(self): |
3cc5663af196
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
39 |
return self.padding-self.counter |
3cc5663af196
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
40 |
|
3cc5663af196
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
41 |
@property |
3cc5663af196
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
42 |
def view(self): |
3cc5663af196
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
43 |
"""this is always an O(1) operation""" |
3cc5663af196
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
44 |
return self.buffer[self.counter:][:self.size] |
3cc5663af196
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
45 |
|
3cc5663af196
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
46 |
def compact(self): |
3cc5663af196
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
47 |
""" |
3cc5663af196
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
48 |
note: only when this function is called, is an O(size) performance hit incurred, |
3cc5663af196
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
49 |
and this cost is amortized over the whole padding space |
3cc5663af196
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
50 |
""" |
3cc5663af196
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
51 |
print 'compacting' |
3cc5663af196
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
52 |
self.buffer[:self.size] = self.view |
3cc5663af196
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
53 |
self.counter = 0 |
3cc5663af196
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
54 |
self.full = True |
3cc5663af196
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
55 |