Sternenlabor/nuki_hub
2
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Arduino Core 3 (#407)

Browse Source 
* Add and remove libs and components for Arduino Core 3

* Arduino Core 3

* Add back Solo1

* Change ESP32-S3 to 4MB build

* Update README.md

* Fix retain and number of retries

* Fix rolling log

* Fix defaults

* Fix BleScanner on Solo1

* Export settings

* Import settings

* Fix HA Battery voltage

* Change submodule

* Update espMqttClient and AsyncTCP

* Webserial and MQTT/Network reconnecting

* Update nuki_ble

---------

Co-authored-by: iranl <iranl@github.com>
This commit is contained in:
iranl
2024-07-05 18:45:39 +02:00
committed by GitHub
parent 193ebb5f91
commit 6b0100fd61
236 changed files with 16390 additions and 9740 deletions
Download Patch File Download Diff File Expand all files Collapse all files

13
lib/espMqttClient/src/Config.h
View File

@@ -60,3 +60,16 @@ the LICENSE file.
#ifndef EMC_USE_WATCHDOG
#define EMC_USE_WATCHDOG 0
#endif
#ifndef EMC_USE_MEMPOOL
#define EMC_USE_MEMPOOL 0
#endif
#if EMC_USE_MEMPOOL
#ifndef EMC_NUM_POOL_ELEMENTS
#define EMC_NUM_POOL_ELEMENTS 32
#endif
#ifndef EMC_SIZE_POOL_ELEMENTS
#define EMC_SIZE_POOL_ELEMENTS 128
#endif
#endif

21
lib/espMqttClient/src/MemoryPool/LICENSE Normal file
View File

@@ -0,0 +1,21 @@
MIT License
Copyright (c) 2024 Bert Melis
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

105
lib/espMqttClient/src/MemoryPool/README.md Normal file
View File

@@ -0,0 +1,105 @@
# Memory Pool
EARLY VERSION. USE AT OWN RISK.
### Description
This is a simple memory pool that doesn't solve the fragmentation problem but contains it. Inside the pool you will still suffer memory fragmentation. The upside is that you're not restricted on memory size. As long as it fits in the pool, you can request any size!
For applications where the (maximum) size to allocate is known, a simple fixed block size memory pool is available. There is no memory fragmentation happening in this case. The downside is wastage of memory if you need less then the specified blocksize.
#### Features
- pool memory is statically allocated
- pool size adjusts on architecture
- no size calculation required: input number of blocks and size of block
- header-only library
- Variable size pool: no restriction on allocated size
- Variable size pool: malloc and free are O(n); The number of allocated blocks affects lookup.
- Fixed size pool: malloc and free are O(1).
[![Test with Platformio](https://github.com/bertmelis/MemoryPool/actions/workflows/test-platformio.yml/badge.svg)](https://github.com/bertmelis/MemoryPool/actions/workflows/test-platformio.yml)
[![cpplint](https://github.com/bertmelis/MemoryPool/actions/workflows/cpplint.yml/badge.svg)](https://github.com/bertmelis/MemoryPool/actions/workflows/cpplint.yml)
<!---[![cppcheck](https://github.com/bertmelis/MemoryPool/actions/workflows/cppcheck.yml/badge.svg)](https://github.com/bertmelis/MemoryPool/actions/workflows/cppcheck.yml)--->
### Usage
#### Variable size pool
```cpp
#include <MemoryPool.h>
Struct MyStruct {
unsigned int id;
std::size_t size;
unsigned char data[256];
};
// pool will be able to hold 10 blocks the size of MyStruct
MemoryPool::Variable<10, sizeof(MyStruct)> pool;
// you can allocate the specified blocksize
// allocation is done in number of 'unsigned char'
MyStruct* s = reinterpret_cast<MyStruct*>(pool.malloc(sizeof(MyStruct)));
// you can allocate less than the specified blocksize
int* i = reinterpret_cast<int*>(pool.malloc(sizeof(int)));
// you can allocate more than the specified blocksize
unsigned char* m = reinterpret_cast<unsigned char*>(pool.malloc(400));
pool.free(s);
pool.free(i);
pool.free(m);
```
#### Fixed size pool
```cpp
#include <MemoryPool.h>
Struct MyStruct {
unsigned int id;
std::size_t size;
unsigned char data[256];
};
// pool will be able to hold 10 blocks the size of MyStruct
MemoryPool::Fixed<10, sizeof(MyStruct)> pool;
// there is no size argument in the malloc function!
MyStruct* s = reinterpret_cast<MyStruct*>(pool.malloc());
// you can allocate less than the specified blocksize
int* i = reinterpret_cast<int*>(pool.malloc());
pool.free(s);
pool.free(i);
```
#### How it works
##### Variable size pool
Free blocks are organized as a linked list with their header (contains pointer to next and size). An allocated block also has this header with it's pointer set to `nullptr`. Therefore, each allocation wastes memory the size of the header (`sizeof(void*) + sizeof(std::size_t)`). On creation, the pool calculations the needed space to store the number of blocks wich each their header.
However, memory allocation isn't restricted the the specified blocksize. So in reality, you can allocate more if you allocate larger chunks because less memory blocks means less headers. After all, memory needs to be contiguous.
If you inspect the pool you'll see that a free pool only has one big block.
Allocation is linear: the pool is iterated until a suitable spot is found.
Freeing is also linear as the pool is traversed to insert the chunk in the linked list of free blocks
When freeing, free blocks which are adjacent are combined into one.
##### Fixed size pool
The fixed size pool is implemented as an array. Free blocks are saved as a linked list in this array.
### Bugs and feature requests
Please use Github's facilities to get in touch.
### License
This library is released under the MIT Licence. A copy is included in the repo.

16
lib/espMqttClient/src/MemoryPool/keywords.txt Normal file
View File

@@ -0,0 +1,16 @@
# Datatypes (KEYWORD1)
Fixed KEYWORD1
Variable KEYWORD1
# Methods and Functions (KEYWORD2)
malloc KEYWORD2
free KEYWORD2
freeMemory KEYWORD2
maxBlockSize KEYWORD2
print KEYWORD2
# Structures (KEYWORD3)
# structure KEYWORD3
# Constants (LITERAL1)
MemoryPool LITERAL1

21
lib/espMqttClient/src/MemoryPool/library.json Normal file
View File

@@ -0,0 +1,21 @@
{
"name": "MemoryPool",
"keywords": "memory",
"description": "A simple memory pool for fixed and variable sizes",
"authors":
{
"name": "Bert Melis",
"url": "https://github.com/bertmelis"
},
"license": "MIT",
"homepage": "https://github.com/bertmelis/MemoryPool",
"repository":
{
"type": "git",
"url": "https://github.com/bertmelis/MemoryPool.git"
},
"version": "0.1.0",
"frameworks": "*",
"platforms": "*",
"headers": ["MemoryPool.h"]
}

10
lib/espMqttClient/src/MemoryPool/library.properties Normal file
View File

@@ -0,0 +1,10 @@
name=MemoryPool
version=0.1.0
author=Bert Melis
maintainer=Bert Melis
sentence=A simple memory pool for fixed and variable sizes
paragraph=
category=Other
url=https://github.com/bertmelis/MemoryPool
architectures=*
includes=MemoryPool.h

119
lib/espMqttClient/src/MemoryPool/src/Fixed.h Normal file
View File

@@ -0,0 +1,119 @@
/*
Copyright (c) 2024 Bert Melis. All rights reserved.
This work is licensed under the terms of the MIT license.
For a copy, see <https://opensource.org/licenses/MIT> or
the LICENSE file.
*/
#pragma once
#include <cstddef> // std::size_t
#include <cassert> // assert
#if _GLIBCXX_HAS_GTHREADS
#include <mutex> // NOLINT [build/c++11] std::mutex, std::lock_guard
#else
#warning "The memory pool is not thread safe"
#endif
#ifdef MEMPOL_DEBUG
#include <iostream>
#endif
namespace MemoryPool {
template <std::size_t nrBlocks, std::size_t blocksize>
class Fixed {
public:
Fixed() // cppcheck-suppress uninitMemberVar
: _buffer{0}
, _head(_buffer) {
unsigned char* b = _head;
std::size_t adjustedBlocksize = sizeof(std::size_t) > blocksize ? sizeof(std::size_t) : blocksize;
for (std::size_t i = 0; i < nrBlocks - 1; ++i) {
*reinterpret_cast<unsigned char**>(b) = b + adjustedBlocksize;
b += adjustedBlocksize;
}
*reinterpret_cast<unsigned char**>(b) = nullptr;
}
// no copy nor move
Fixed (const Fixed&) = delete;
Fixed& operator= (const Fixed&) = delete;
void* malloc() {
#if _GLIBCXX_HAS_GTHREADS
const std::lock_guard<std::mutex> lockGuard(_mutex);
#endif
if (_head) {
void* retVal = _head;
_head = *reinterpret_cast<unsigned char**>(_head);
return retVal;
}
return nullptr;
}
void free(void* ptr) {
if (!ptr) return;
#if _GLIBCXX_HAS_GTHREADS
const std::lock_guard<std::mutex> lockGuard(_mutex);
#endif
*reinterpret_cast<unsigned char**>(ptr) = _head;
_head = reinterpret_cast<unsigned char*>(ptr);
}
std::size_t freeMemory() {
#if _GLIBCXX_HAS_GTHREADS
const std::lock_guard<std::mutex> lockGuard(_mutex);
#endif
unsigned char* i = _head;
std::size_t retVal = 0;
while (i) {
retVal += blocksize;
i = reinterpret_cast<unsigned char**>(i)[0];
}
return retVal;
}
#ifdef MEMPOL_DEBUG
void print() {
std::size_t adjustedBlocksize = sizeof(std::size_t) > blocksize ? sizeof(std::size_t) : blocksize;
std::cout << "+--------------------" << std::endl;
std::cout << "|start:" << reinterpret_cast<void*>(_buffer) << std::endl;
std::cout << "|blocks:" << nrBlocks << std::endl;
std::cout << "|blocksize:" << adjustedBlocksize << std::endl;
std::cout << "|head: " << reinterpret_cast<void*>(_head) << std::endl;
unsigned char* currentBlock = _buffer;
for (std::size_t i = 0; i < nrBlocks; ++i) {
std::cout << "|" << i + 1 << ": " << reinterpret_cast<void*>(currentBlock) << std::endl;
if (_isFree(currentBlock)) {
std::cout << "| free" << std::endl;
std::cout << "| next: " << reinterpret_cast<void*>(*reinterpret_cast<unsigned char**>(currentBlock)) << std::endl;
} else {
std::cout << "| allocated" << std::endl;
}
currentBlock += adjustedBlocksize;
}
std::cout << "+--------------------" << std::endl;
}
bool _isFree(const unsigned char* ptr) {
unsigned char* b = _head;
while (b) {
if (b == ptr) return true;
b = *reinterpret_cast<unsigned char**>(b);
}
return false;
}
#endif
private:
unsigned char _buffer[nrBlocks * (sizeof(std::size_t) > blocksize ? sizeof(std::size_t) : blocksize)];
unsigned char* _head;
#if _GLIBCXX_HAS_GTHREADS
std::mutex _mutex;
#endif
};
} // end namespace MemoryPool

12
lib/espMqttClient/src/MemoryPool/src/MemoryPool.h Normal file
View File

@@ -0,0 +1,12 @@
/*
Copyright (c) 2024 Bert Melis. All rights reserved.
This work is licensed under the terms of the MIT license.
For a copy, see <https://opensource.org/licenses/MIT> or
the LICENSE file.
*/
#pragma once
#include "Variable.h"
#include "Fixed.h"

242
lib/espMqttClient/src/MemoryPool/src/Variable.h Normal file
View File

@@ -0,0 +1,242 @@
/*
Copyright (c) 2024 Bert Melis. All rights reserved.
This work is licensed under the terms of the MIT license.
For a copy, see <https://opensource.org/licenses/MIT> or
the LICENSE file.
*/
#pragma once
#include <cstddef> // std::size_t
#include <cassert> // assert
#if _GLIBCXX_HAS_GTHREADS
#include <mutex> // NOLINT [build/c++11] std::mutex, std::lock_guard
#else
#warning "The memory pool is not thread safe"
#endif
#ifdef MEMPOL_DEBUG
#include <iostream>
#endif
namespace MemoryPool {
template <std::size_t nrBlocks, std::size_t blocksize>
class Variable {
public:
Variable()
: _buffer{0}
, _head(nullptr)
#ifdef MEMPOL_DEBUG
, _bufferSize(0)
#endif
{
std::size_t _normBlocksize = blocksize / sizeof(BlockHeader) + ((blocksize % sizeof(BlockHeader)) ? 1 : 0);
size_t nrBlocksToAlloc = nrBlocks * (_normBlocksize + 1);
BlockHeader* h = reinterpret_cast<BlockHeader*>(_buffer);
h->next = nullptr;
h->size = nrBlocksToAlloc;
_head = h;
#ifdef MEMPOL_DEBUG
_bufferSize = nrBlocksToAlloc;
#endif
}
// no copy nor move
Variable (const Variable&) = delete;
Variable& operator= (const Variable&) = delete;
void* malloc(size_t size) {
#if _GLIBCXX_HAS_GTHREADS
const std::lock_guard<std::mutex> lockGuard(_mutex);
#endif
if (size == 0) return nullptr;
size = (size / sizeof(BlockHeader) + (size % sizeof(BlockHeader) != 0)) + 1; // count by BlockHeader size, add 1 for header
#ifdef MEMPOL_DEBUG
std::cout << "malloc (raw) " << size << std::endl;
std::cout << "malloc (adj) " << size << " - ";
#endif
BlockHeader* currentBlock = _head;
BlockHeader* previousBlock = nullptr;
void* retVal = nullptr;
// iterate through linked free blocks
while (currentBlock) {
// consume whole block is size equals required size
if (currentBlock->size == size) {
if (previousBlock) previousBlock->next = currentBlock->next;
break;
// split block if size is larger and add second part to list of free blocks
} else if (currentBlock->size > size) {
BlockHeader* newBlock = currentBlock + size;
if (previousBlock) previousBlock->next = newBlock;
newBlock->next = currentBlock->next;
newBlock->size = currentBlock->size - size;
currentBlock->next = newBlock;
break;
}
previousBlock = currentBlock;
currentBlock = currentBlock->next;
}
if (currentBlock) {
if (currentBlock == _head) {
_head = currentBlock->next;
}
currentBlock->size = size;
currentBlock->next = nullptr; // used when freeing memory
retVal = currentBlock + 1;
#ifdef MEMPOL_DEBUG
std::cout << "ok" << std::endl;
#endif
} else {
#ifdef MEMPOL_DEBUG
std::cout << "nok" << std::endl;
#endif
(void)0;
}
return retVal;
}
void free(void* ptr) {
if (!ptr) return;
// check if ptr points to region in _buffer
#ifdef MEMPOL_DEBUG
std::cout << "free " << static_cast<void*>(reinterpret_cast<BlockHeader*>(ptr) - 1) << std::endl;
#endif
#if _GLIBCXX_HAS_GTHREADS
const std::lock_guard<std::mutex> lockGuard(_mutex);
#endif
BlockHeader* toFree = reinterpret_cast<BlockHeader*>(ptr) - 1;
BlockHeader* previous = reinterpret_cast<BlockHeader*>(_buffer);
BlockHeader* next = _head;
// toFree is the only free block
if (!next) {
_head = toFree;
return;
}
while (previous) {
if (!next || toFree < next) {
// 1. add block to linked list of free blocks
if (toFree < _head) {
toFree->next = _head;
_head = toFree;
} else {
previous->next = toFree;
toFree->next = next;
}
// 2. merge with previous if adjacent
if (toFree > _head && toFree == previous + previous->size) {
previous->size += toFree->size;
previous->next = toFree->next;
toFree = previous; // used in next check
}
// 3. merge with next if adjacent
if (toFree + toFree->size == next) {
toFree->size += next->size;
toFree->next = next->next;
}
// 4. done
return;
}
previous = next;
next = next->next;
}
}
std::size_t freeMemory() {
#if _GLIBCXX_HAS_GTHREADS
const std::lock_guard<std::mutex> lockGuard(_mutex);
#endif
size_t retVal = 0;
BlockHeader* currentBlock = reinterpret_cast<BlockHeader*>(_head);
while (currentBlock) {
retVal += currentBlock->size - 1;
currentBlock = currentBlock->next;
}
return retVal * sizeof(BlockHeader);
}
std::size_t maxBlockSize() {
#if _GLIBCXX_HAS_GTHREADS
const std::lock_guard<std::mutex> lockGuard(_mutex);
#endif
size_t retVal = 0;
BlockHeader* currentBlock = reinterpret_cast<BlockHeader*>(_head);
while (currentBlock) {
retVal = (currentBlock->size - 1 > retVal) ? currentBlock->size - 1 : retVal;
currentBlock = currentBlock->next;
}
return retVal * sizeof(BlockHeader);
}
#ifdef MEMPOL_DEBUG
void print() {
std::cout << "+--------------------" << std::endl;
std::cout << "|start:" << static_cast<void*>(_buffer) << std::endl;
std::cout << "|size:" << _bufferSize << std::endl;
std::cout << "|headersize:" << sizeof(BlockHeader) << std::endl;
std::cout << "|head: " << static_cast<void*>(_head) << std::endl;
BlockHeader* nextFreeBlock = _head;
BlockHeader* currentBlock = reinterpret_cast<BlockHeader*>(_buffer);
size_t blockNumber = 1;
while (currentBlock < reinterpret_cast<BlockHeader*>(_buffer) + _bufferSize) {
std::cout << "|" << blockNumber << ": " << static_cast<void*>(currentBlock) << std::endl;
std::cout << "| " << static_cast<void*>(currentBlock->next) << std::endl;
std::cout << "| " << currentBlock->size << std::endl;
if (currentBlock == nextFreeBlock) {
std::cout << "| free" << std::endl;
nextFreeBlock = nextFreeBlock->next;
} else {
std::cout << "| allocated" << std::endl;
}
++blockNumber;
currentBlock += currentBlock->size;
}
std::cout << "+--------------------" << std::endl;
}
#endif
private:
struct BlockHeader {
BlockHeader* next;
std::size_t size;
};
/*
pool size is aligned to sizeof(BlockHeader).
requested blocksize is therefore multiple of blockheader (rounded up)
total size = nr requested blocks * multiplier * blockheadersize
see constructor for calculation
*/
unsigned char _buffer[(nrBlocks * ((blocksize / sizeof(BlockHeader) + ((blocksize % sizeof(BlockHeader)) ? 1 : 0)) + 1)) * sizeof(BlockHeader)];
BlockHeader* _head;
#if _GLIBCXX_HAS_GTHREADS
std::mutex _mutex;
#endif
#ifdef MEMPOL_DEBUG
std::size_t _bufferSize;
#endif
};
} // end namespace MemoryPool

97
lib/espMqttClient/src/MqttClient.cpp
View File

@@ -118,9 +118,10 @@ bool MqttClient::connect() {
}
#endif
} else {
EMC_SEMAPHORE_GIVE();
emc_log_e("Could not create CONNECT packet");
EMC_SEMAPHORE_GIVE();
_onError(0, Error::OUT_OF_MEMORY);
EMC_SEMAPHORE_TAKE();
}
EMC_SEMAPHORE_GIVE();
}
@@ -151,7 +152,9 @@ uint16_t MqttClient::publish(const char* topic, uint8_t qos, bool retain, const
uint16_t packetId = (qos > 0) ? _getNextPacketId() : 1;
if (!_addPacket(packetId, topic, payload, length, qos, retain)) {
emc_log_e("Could not create PUBLISH packet");
EMC_SEMAPHORE_GIVE();
_onError(packetId, Error::OUT_OF_MEMORY);
EMC_SEMAPHORE_TAKE();
packetId = 0;
}
EMC_SEMAPHORE_GIVE();
@@ -175,7 +178,9 @@ uint16_t MqttClient::publish(const char* topic, uint8_t qos, bool retain, espMqt
uint16_t packetId = (qos > 0) ? _getNextPacketId() : 1;
if (!_addPacket(packetId, topic, callback, length, qos, retain)) {
emc_log_e("Could not create PUBLISH packet");
EMC_SEMAPHORE_GIVE();
_onError(packetId, Error::OUT_OF_MEMORY);
EMC_SEMAPHORE_TAKE();
packetId = 0;
}
EMC_SEMAPHORE_GIVE();
@@ -183,7 +188,9 @@ uint16_t MqttClient::publish(const char* topic, uint8_t qos, bool retain, espMqt
}
void MqttClient::clearQueue(bool deleteSessionData) {
EMC_SEMAPHORE_TAKE();
_clearQueue(deleteSessionData ? 2 : 0);
EMC_SEMAPHORE_GIVE();
}
const char* MqttClient::getClientId() const {
@@ -222,14 +229,19 @@ void MqttClient::loop() {
_parser.reset();
_lastClientActivity = _lastServerActivity = millis();
_setState(State::connectingMqtt);
} else if (_transport->disconnected()) { // sync: implemented as "not connected"; async: depending on state of pcb in underlying lib
_setState(State::disconnectingTcp1);
_disconnectReason = DisconnectReason::TCP_DISCONNECTED;
}
break;
case State::connectingMqtt:
#if EMC_WAIT_FOR_CONNACK
if (_transport->connected()) {
EMC_SEMAPHORE_TAKE();
_sendPacket();
_checkIncoming();
_checkPing();
EMC_SEMAPHORE_GIVE();
} else {
_setState(State::disconnectingTcp1);
_disconnectReason = DisconnectReason::TCP_DISCONNECTED;
@@ -246,10 +258,12 @@ void MqttClient::loop() {
case State::disconnectingMqtt2:
if (_transport->connected()) {
// CONNECT packet is first in the queue
EMC_SEMAPHORE_TAKE();
_checkOutbox();
_checkIncoming();
_checkPing();
_checkTimeout();
EMC_SEMAPHORE_GIVE();
} else {
_setState(State::disconnectingTcp1);
_disconnectReason = DisconnectReason::TCP_DISCONNECTED;
@@ -262,15 +276,16 @@ void MqttClient::loop() {
EMC_SEMAPHORE_GIVE();
emc_log_e("Could not create DISCONNECT packet");
_onError(0, Error::OUT_OF_MEMORY);
EMC_SEMAPHORE_TAKE();
} else {
_setState(State::disconnectingMqtt2);
}
}
EMC_SEMAPHORE_GIVE();
_checkOutbox();
_checkIncoming();
_checkPing();
_checkTimeout();
EMC_SEMAPHORE_GIVE();
break;
case State::disconnectingTcp1:
_transport->stop();
@@ -278,10 +293,14 @@ void MqttClient::loop() {
break; // keep break to accomodate async clients
case State::disconnectingTcp2:
if (_transport->disconnected()) {
EMC_SEMAPHORE_TAKE();
_clearQueue(0);
EMC_SEMAPHORE_GIVE();
_bytesSent = 0;
_setState(State::disconnected);
if (_onDisconnectCallback) _onDisconnectCallback(_disconnectReason);
if (_onDisconnectCallback) {
_onDisconnectCallback(_disconnectReason);
}
}
break;
// all cases covered, no default case
@@ -332,14 +351,12 @@ void MqttClient::_checkOutbox() {
}
int MqttClient::_sendPacket() {
EMC_SEMAPHORE_TAKE();
OutgoingPacket* packet = _outbox.getCurrent();
size_t written = 0;
if (packet) {
size_t wantToWrite = packet->packet.available(_bytesSent);
if (wantToWrite == 0) {
EMC_SEMAPHORE_GIVE();
return 0;
}
written = _transport->write(packet->packet.data(_bytesSent), wantToWrite);
@@ -348,12 +365,10 @@ int MqttClient::_sendPacket() {
_bytesSent += written;
emc_log_i("tx %zu/%zu (%02x)", _bytesSent, packet->packet.size(), packet->packet.packetType());
}
EMC_SEMAPHORE_GIVE();
return written;
}
bool MqttClient::_advanceOutbox() {
EMC_SEMAPHORE_TAKE();
OutgoingPacket* packet = _outbox.getCurrent();
if (packet && _bytesSent == packet->packet.size()) {
if ((packet->packet.packetType()) == PacketType.DISCONNECT) {
@@ -370,7 +385,6 @@ bool MqttClient::_advanceOutbox() {
packet = _outbox.getCurrent();
_bytesSent = 0;
}
EMC_SEMAPHORE_GIVE();
return packet;
}
@@ -390,7 +404,7 @@ void MqttClient::_checkIncoming() {
_setState(State::disconnectingTcp1);
return;
}
switch (packetType & 0xF0) {
switch (packetType) {
case PacketType.CONNACK:
_onConnack();
if (_state != State::connected) {
@@ -455,19 +469,15 @@ void MqttClient::_checkPing() {
if (!_pingSent &&
((currentMillis - _lastClientActivity > _keepAlive) ||
(currentMillis - _lastServerActivity > _keepAlive))) {
EMC_SEMAPHORE_TAKE();
if (!_addPacket(PacketType.PINGREQ)) {
EMC_SEMAPHORE_GIVE();
emc_log_e("Could not create PING packet");
return;
}
EMC_SEMAPHORE_GIVE();
_pingSent = true;
}
}
void MqttClient::_checkTimeout() {
EMC_SEMAPHORE_TAKE();
espMqttClientInternals::Outbox<OutgoingPacket>::Iterator it = _outbox.front();
// check that we're not busy sending
// don't check when first item hasn't been sent yet
@@ -477,7 +487,6 @@ void MqttClient::_checkTimeout() {
_outbox.resetCurrent();
}
}
EMC_SEMAPHORE_GIVE();
}
void MqttClient::_onConnack() {
@@ -489,7 +498,9 @@ void MqttClient::_onConnack() {
_clearQueue(1);
}
if (_onConnectCallback) {
EMC_SEMAPHORE_GIVE();
_onConnectCallback(_parser.getPacket().variableHeader.fixed.connackVarHeader.sessionPresent);
EMC_SEMAPHORE_TAKE();
}
} else {
_setState(State::disconnectingTcp1);
@@ -507,14 +518,11 @@ void MqttClient::_onPublish() {
bool callback = true;
if (qos == 1) {
if (p.payload.index + p.payload.length == p.payload.total) {
EMC_SEMAPHORE_TAKE();
if (!_addPacket(PacketType.PUBACK, packetId)) {
emc_log_e("Could not create PUBACK packet");
}
EMC_SEMAPHORE_GIVE();
}
} else if (qos == 2) {
EMC_SEMAPHORE_TAKE();
espMqttClientInternals::Outbox<OutgoingPacket>::Iterator it = _outbox.front();
while (it) {
if ((it.get()->packet.packetType()) == PacketType.PUBREC && it.get()->packet.packetId() == packetId) {
@@ -529,20 +537,22 @@ void MqttClient::_onPublish() {
emc_log_e("Could not create PUBREC packet");
}
}
EMC_SEMAPHORE_GIVE();
}
if (callback && _onMessageCallback) _onMessageCallback({qos, dup, retain, packetId},
p.variableHeader.topic,
p.payload.data,
p.payload.length,
p.payload.index,
p.payload.total);
if (callback && _onMessageCallback) {
EMC_SEMAPHORE_GIVE();
_onMessageCallback({qos, dup, retain, packetId},
p.variableHeader.topic,
p.payload.data,
p.payload.length,
p.payload.index,
p.payload.total);
EMC_SEMAPHORE_TAKE();
}
}
void MqttClient::_onPuback() {
bool callback = false;
uint16_t idToMatch = _parser.getPacket().variableHeader.fixed.packetId;
EMC_SEMAPHORE_TAKE();
espMqttClientInternals::Outbox<OutgoingPacket>::Iterator it = _outbox.front();
while (it) {
// PUBACKs come in the order PUBs are sent. So we only check the first PUB packet in outbox
@@ -558,9 +568,12 @@ void MqttClient::_onPuback() {
}
++it;
}
EMC_SEMAPHORE_GIVE();
if (callback) {
if (_onPublishCallback) _onPublishCallback(idToMatch);
if (_onPublishCallback) {
EMC_SEMAPHORE_GIVE();
_onPublishCallback(idToMatch);
EMC_SEMAPHORE_TAKE();
}
} else {
emc_log_w("No matching PUBLISH packet found");
}
@@ -569,7 +582,6 @@ void MqttClient::_onPuback() {
void MqttClient::_onPubrec() {
bool success = false;
uint16_t idToMatch = _parser.getPacket().variableHeader.fixed.packetId;
EMC_SEMAPHORE_TAKE();
espMqttClientInternals::Outbox<OutgoingPacket>::Iterator it = _outbox.front();
while (it) {
// PUBRECs come in the order PUBs are sent. So we only check the first PUB packet in outbox
@@ -591,13 +603,11 @@ void MqttClient::_onPubrec() {
if (!success) {
emc_log_w("No matching PUBLISH packet found");
}
EMC_SEMAPHORE_GIVE();
}
void MqttClient::_onPubrel() {
bool success = false;
uint16_t idToMatch = _parser.getPacket().variableHeader.fixed.packetId;
EMC_SEMAPHORE_TAKE();
espMqttClientInternals::Outbox<OutgoingPacket>::Iterator it = _outbox.front();
while (it) {
// PUBRELs come in the order PUBRECs are sent. So we only check the first PUBREC packet in outbox
@@ -619,12 +629,10 @@ void MqttClient::_onPubrel() {
if (!success) {
emc_log_w("No matching PUBREC packet found");
}
EMC_SEMAPHORE_GIVE();
}
void MqttClient::_onPubcomp() {
bool callback = false;
EMC_SEMAPHORE_TAKE();
espMqttClientInternals::Outbox<OutgoingPacket>::Iterator it = _outbox.front();
uint16_t idToMatch = _parser.getPacket().variableHeader.fixed.packetId;
while (it) {
@@ -641,9 +649,12 @@ void MqttClient::_onPubcomp() {
}
++it;
}
EMC_SEMAPHORE_GIVE();
if (callback) {
if (_onPublishCallback) _onPublishCallback(idToMatch);
if (_onPublishCallback) {
EMC_SEMAPHORE_GIVE();
_onPublishCallback(idToMatch);
EMC_SEMAPHORE_TAKE();
}
} else {
emc_log_w("No matching PUBREL packet found");
}
@@ -652,7 +663,6 @@ void MqttClient::_onPubcomp() {
void MqttClient::_onSuback() {
bool callback = false;
uint16_t idToMatch = _parser.getPacket().variableHeader.fixed.packetId;
EMC_SEMAPHORE_TAKE();
espMqttClientInternals::Outbox<OutgoingPacket>::Iterator it = _outbox.front();
while (it) {
if (((it.get()->packet.packetType()) == PacketType.SUBSCRIBE) && it.get()->packet.packetId() == idToMatch) {
@@ -662,9 +672,12 @@ void MqttClient::_onSuback() {
}
++it;
}
EMC_SEMAPHORE_GIVE();
if (callback) {
if (_onSubscribeCallback) _onSubscribeCallback(idToMatch, reinterpret_cast<const espMqttClientTypes::SubscribeReturncode*>(_parser.getPacket().payload.data), _parser.getPacket().payload.total);
if (_onSubscribeCallback) {
EMC_SEMAPHORE_GIVE();
_onSubscribeCallback(idToMatch, reinterpret_cast<const espMqttClientTypes::SubscribeReturncode*>(_parser.getPacket().payload.data), _parser.getPacket().payload.total);
EMC_SEMAPHORE_TAKE();
}
} else {
emc_log_w("received SUBACK without SUB");
}
@@ -672,7 +685,6 @@ void MqttClient::_onSuback() {
void MqttClient::_onUnsuback() {
bool callback = false;
EMC_SEMAPHORE_TAKE();
espMqttClientInternals::Outbox<OutgoingPacket>::Iterator it = _outbox.front();
uint16_t idToMatch = _parser.getPacket().variableHeader.fixed.packetId;
while (it) {
@@ -683,9 +695,12 @@ void MqttClient::_onUnsuback() {
}
++it;
}
EMC_SEMAPHORE_GIVE();
if (callback) {
if (_onUnsubscribeCallback) _onUnsubscribeCallback(idToMatch);
if (_onUnsubscribeCallback) {
EMC_SEMAPHORE_GIVE();
_onUnsubscribeCallback(idToMatch);
EMC_SEMAPHORE_TAKE();
}
} else {
emc_log_w("received UNSUBACK without UNSUB");
}
@@ -693,7 +708,6 @@ void MqttClient::_onUnsuback() {
void MqttClient::_clearQueue(int clearData) {
emc_log_i("clearing queue (clear session: %d)", clearData);
EMC_SEMAPHORE_TAKE();
espMqttClientInternals::Outbox<OutgoingPacket>::Iterator it = _outbox.front();
if (clearData == 0) {
// keep PUB (qos > 0, aka packetID != 0), PUBREC and PUBREL
@@ -723,7 +737,6 @@ void MqttClient::_clearQueue(int clearData) {
_outbox.remove(it);
}
}
EMC_SEMAPHORE_GIVE();
}
void MqttClient::_onError(uint16_t packetId, espMqttClientTypes::Error error) {

10
lib/espMqttClient/src/MqttClient.h
View File

@@ -32,11 +32,12 @@ class MqttClient {
bool disconnect(bool force = false);
template <typename... Args>
uint16_t subscribe(const char* topic, uint8_t qos, Args&&... args) {
uint16_t packetId = _getNextPacketId();
uint16_t packetId = 0;
if (_state != State::connected) {
packetId = 0;
return packetId;
} else {
EMC_SEMAPHORE_TAKE();
packetId = _getNextPacketId();
if (!_addPacket(packetId, topic, qos, std::forward<Args>(args) ...)) {
emc_log_e("Could not create SUBSCRIBE packet");
packetId = 0;
@@ -47,11 +48,12 @@ class MqttClient {
}
template <typename... Args>
uint16_t unsubscribe(const char* topic, Args&&... args) {
uint16_t packetId = _getNextPacketId();
uint16_t packetId = 0;
if (_state != State::connected) {
packetId = 0;
return packetId;
} else {
EMC_SEMAPHORE_TAKE();
packetId = _getNextPacketId();
if (!_addPacket(packetId, topic, std::forward<Args>(args) ...)) {
emc_log_e("Could not create UNSUBSCRIBE packet");
packetId = 0;

48
lib/espMqttClient/src/Outbox.h
View File

@@ -9,7 +9,12 @@ the LICENSE file.
#pragma once
#include <new> // new (std::nothrow)
#if EMC_USE_MEMPOOL
#include "MemoryPool/src/MemoryPool.h"
#include "Config.h"
#else
#include <new> // new (std::nothrow)
#endif
#include <utility> // std::forward
namespace espMqttClientInternals {
@@ -28,11 +33,20 @@ class Outbox {
: _first(nullptr)
, _last(nullptr)
, _current(nullptr)
, _prev(nullptr) {}
, _prev(nullptr)
#if EMC_USE_MEMPOOL
, _memPool()
#endif
{}
~Outbox() {
while (_first) {
Node* n = _first->next;
#if EMC_USE_MEMPOOL
_first->~Node();
_memPool.free(_first);
#else
delete _first;
#endif
_first = n;
}
}
@@ -79,7 +93,15 @@ class Outbox {
template <class... Args>
Iterator emplace(Args&&... args) {
Iterator it;
Node* node = new (std::nothrow) Node(std::forward<Args>(args) ...);
#if EMC_USE_MEMPOOL
void* buf = _memPool.malloc();
Node* node = nullptr;
if (buf) {
node = new(buf) Node(std::forward<Args>(args) ...);
}
#else
Node* node = new(std::nothrow) Node(std::forward<Args>(args) ...);
#endif
if (node != nullptr) {
if (!_first) {
// queue is empty
@@ -103,7 +125,15 @@ class Outbox {
template <class... Args>
Iterator emplaceFront(Args&&... args) {
Iterator it;
Node* node = new (std::nothrow) Node(std::forward<Args>(args) ...);
#if EMC_USE_MEMPOOL
void* buf = _memPool.malloc();
Node* node = nullptr;
if (buf) {
node = new(buf) Node(std::forward<Args>(args) ...);
}
#else
Node* node = new(std::nothrow) Node(std::forward<Args>(args) ...);
#endif
if (node != nullptr) {
if (!_first) {
// queue is empty
@@ -178,6 +208,9 @@ class Outbox {
Node* _last;
Node* _current;
Node* _prev; // element just before _current
#if EMC_USE_MEMPOOL
MemoryPool::Fixed<EMC_NUM_POOL_ELEMENTS, sizeof(Node)> _memPool;
#endif
void _remove(Node* prev, Node* node) {
if (!node) return;
@@ -210,7 +243,12 @@ class Outbox {
}
// finally, delete the node
delete node;
#if EMC_USE_MEMPOOL
node->~Node();
_memPool.free(node);
#else
delete node;
#endif
}
};

28
lib/espMqttClient/src/Packets/Packet.cpp
View File

@@ -10,8 +10,16 @@ the LICENSE file.
namespace espMqttClientInternals {
#if EMC_USE_MEMPOOL
MemoryPool::Variable<EMC_NUM_POOL_ELEMENTS, EMC_SIZE_POOL_ELEMENTS> Packet::_memPool;
#endif
Packet::~Packet() {
#if EMC_USE_MEMPOOL
_memPool.free(_data);
#else
free(_data);
#endif
}
size_t Packet::available(size_t index) {
@@ -178,7 +186,7 @@ Packet::Packet(espMqttClientTypes::Error& error,
_packetId = 0;
}
if (!_allocate(remainingLength)) {
if (!_allocate(remainingLength, true)) {
error = espMqttClientTypes::Error::OUT_OF_MEMORY;
return;
}
@@ -215,7 +223,7 @@ Packet::Packet(espMqttClientTypes::Error& error,
_packetId = 0;
}
if (!_allocate(remainingLength - payloadLength + std::min(payloadLength, static_cast<size_t>(EMC_RX_BUFFER_SIZE)))) {
if (!_allocate(remainingLength - payloadLength + std::min(payloadLength, static_cast<size_t>(EMC_RX_BUFFER_SIZE)), true)) {
error = espMqttClientTypes::Error::OUT_OF_MEMORY;
return;
}
@@ -251,7 +259,7 @@ Packet::Packet(espMqttClientTypes::Error& error, MQTTPacketType type, uint16_t p
, _payloadStartIndex(0)
, _payloadEndIndex(0)
, _getPayload(nullptr) {
if (!_allocate(2)) {
if (!_allocate(2, true)) {
error = espMqttClientTypes::Error::OUT_OF_MEMORY;
return;
}
@@ -290,7 +298,7 @@ Packet::Packet(espMqttClientTypes::Error& error, MQTTPacketType type)
, _payloadStartIndex(0)
, _payloadEndIndex(0)
, _getPayload(nullptr) {
if (!_allocate(0)) {
if (!_allocate(0, true)) {
error = espMqttClientTypes::Error::OUT_OF_MEMORY;
return;
}
@@ -301,12 +309,20 @@ Packet::Packet(espMqttClientTypes::Error& error, MQTTPacketType type)
bool Packet::_allocate(size_t remainingLength, bool check) {
#if EMC_USE_MEMPOOL
(void) check;
#else
if (check && EMC_GET_FREE_MEMORY() < EMC_MIN_FREE_MEMORY) {
emc_log_w("Packet buffer not allocated: low memory");
return false;
}
#endif
_size = 1 + remainingLengthLength(remainingLength) + remainingLength;
#if EMC_USE_MEMPOOL
_data = reinterpret_cast<uint8_t*>(_memPool.malloc(_size));
#else
_data = reinterpret_cast<uint8_t*>(malloc(_size));
#endif
if (!_data) {
_size = 0;
emc_log_w("Alloc failed (l:%zu)", _size);
@@ -357,7 +373,7 @@ void Packet::_createSubscribe(espMqttClientTypes::Error& error,
size_t remainingLength = 2 + payload; // packetId + payload
// allocate memory
if (!_allocate(remainingLength)) {
if (!_allocate(remainingLength, true)) {
error = espMqttClientTypes::Error::OUT_OF_MEMORY;
return;
}
@@ -387,7 +403,7 @@ void Packet::_createUnsubscribe(espMqttClientTypes::Error& error,
size_t remainingLength = 2 + payload; // packetId + payload
// allocate memory
if (!_allocate(remainingLength)) {
if (!_allocate(remainingLength, true)) {
error = espMqttClientTypes::Error::OUT_OF_MEMORY;
return;
}

10
lib/espMqttClient/src/Packets/Packet.h
View File

@@ -19,6 +19,10 @@ the LICENSE file.
#include "RemainingLength.h"
#include "StringUtil.h"
#if EMC_USE_MEMPOOL
#include "MemoryPool/src/MemoryPool.h"
#endif
namespace espMqttClientInternals {
class Packet {
@@ -133,7 +137,7 @@ class Packet {
private:
// pass remainingLength = total size - header - remainingLengthLength!
bool _allocate(size_t remainingLength, bool check = true);
bool _allocate(size_t remainingLength, bool check);
// fills header and returns index of next available byte in buffer
size_t _fillPublishHeader(uint16_t packetId,
@@ -150,6 +154,10 @@ class Packet {
size_t _chunkedAvailable(size_t index);
const uint8_t* _chunkedData(size_t index) const;
#if EMC_USE_MEMPOOL
static MemoryPool::Variable<EMC_NUM_POOL_ELEMENTS, EMC_SIZE_POOL_ELEMENTS> _memPool;
#endif
};
} // end namespace espMqttClientInternals