Merge pull request #1519 from nodemcu/dev

Next 1.5.4.1 master drop

Author: devsaurus

.github/ISSUE_TEMPLATE.md

@@ -7,7 +7,7 @@ Use one of the two templates below and delete the rest.
 ### Actual behavior
 
 ### Test code
-Provide a minimal and isolated test which will reproduce the problem.
+Provide a [Minimal, Complete, and Verifiable example](http://stackoverflow.com/help/mcve) which will reproduce the problem.
 ```Lua
 -- add code here
 ```

.github/PULL_REQUEST_TEMPLATE.md

@@ -2,10 +2,9 @@ Fixes #\<GitHub-issue-number\>.
 
 Make sure all boxes are checked (add x inside the brackets) when you submit your contribution, remove this sentence before doing so.
 
-- [ ] This PR is compliant with the [contributing guidelines](https://github.com/nodemcu/nodemcu-firmware/blob/dev/CONTRIBUTING.md) (if not, please describe why).
+- [ ] This PR is for the `dev` branch rather than for `master`.
+- [ ] This PR is compliant with the [other contributing guidelines](https://github.com/nodemcu/nodemcu-firmware/blob/dev/CONTRIBUTING.md) as well (if not, please describe why).
 - [ ] I have thoroughly tested my contribution.
 - [ ] The code changes are reflected in the documentation at `docs/en/*`.
 
-\<Description of and rational behind this PR\>
-
-Committers supporting this PR: leave blank
+\<Description of and rational behind this PR\>

CONTRIBUTING.md

@@ -4,7 +4,7 @@
 
 The following is a set of guidelines for contributing to NodeMCU on GitHub. These are just guidelines, not rules, use your best judgment and feel free to propose changes to this document in a pull request.
 
-It is appreciated but optional if you raise an issue _before_ you start changing NodeMCU, discussing the proposed change; emphasising that the you are proposing to develop the patch yourself, and outling the strategy for implementation. This type of discussion is what we should be doing on the issues list and it is better to do this before or in parallel to developing the patch rather than having "you should have done it this way" type of feedback on the PR itself.
+It is appreciated but optional if you raise an issue _before_ you start changing NodeMCU, discussing the proposed change; emphasing that the you are proposing to develop the patch yourself, and outlining the strategy for implementation. This type of discussion is what we should be doing on the issues list and it is better to do this before or in parallel to developing the patch rather than having "you should have done it this way" type of feedback on the PR itself.
 
 ### Table Of Contents
 
@@ -17,7 +17,7 @@ It is appreciated but optional if you raise an issue _before_ you start changing
 
 ## Development environment setup
 Use the platform and tools you feel most comfortable with. There are  no constraints imposed by this project. You have (at least) two options to set up the toolchain to build the NodeMCU firmware:
-- [Full-fledged Linux enviroment](http://www.esp8266.com/wiki/doku.php?id=toolchain#how_to_setup_a_vm_to_host_your_toolchain), either physical or virtual.
+- [Full-fledged Linux environment](http://www.esp8266.com/wiki/doku.php?id=toolchain#how_to_setup_a_vm_to_host_your_toolchain), either physical or virtual.
 - [Docker image](https://hub.docker.com/r/marcelstoer/nodemcu-build/) which allows to run the build inside the container as if you were running a build script on your local machine.
 
 ## Writing Documentation
@@ -32,15 +32,15 @@ There are essentially only two things to keep in mind if you're contributing a P
 
 If you also want to verify that all is well with your Markdown files you can install Python-based [MkDocs](http://www.mkdocs.org/), which is used by RTD to build the static HTML files, and run `mkdocs serve` in the root of your NodeMCU firmware directory.
 
-A note on Markdown *syntax*. As Mkdocs is Python-based it's no surprise it uses a [Python Markdown implementation](https://pythonhosted.org/Markdown/). The good news is that it sticks pretty closely to John Gruber's Markdown and also [supports tables and fenced code blocks](http://www.mkdocs.org/user-guide/writing-your-docs/#markdown-extensions) just like GitHub does.
+A note on Markdown *syntax*. As MkDocs is Python-based it's no surprise it uses a [Python Markdown implementation](https://pythonhosted.org/Markdown/). The good news is that it sticks pretty closely to John Gruber's Markdown and also [supports tables and fenced code blocks](http://www.mkdocs.org/user-guide/writing-your-docs/#markdown-extensions) just like GitHub does.
 
 A collection of doc-writing hints and tips is maintained on the [wiki](https://github.com/nodemcu/nodemcu-firmware/wiki/Notes-about-writing-docs).
 
 If you're interested in some NodeMCU history you're welcome to read [issue #774](https://github.com/nodemcu/nodemcu-firmware/issues/774)
 
 ## Working with Git and GitHub
 
-*Pull requests for new features and major fixes should be opened against the `dev` branch.*
+*Pull requests for new features and fixes should be opened against the `dev` branch.*
 
 Avoid intermediate merge commits. [Rebase](https://www.atlassian.com/git/tutorials/merging-vs-rebasing) your feature branch onto `dev` to pull updates and verify your local changes against them before placing the pull request.
 
@@ -52,12 +52,12 @@ Avoid intermediate merge commits. [Rebase](https://www.atlassian.com/git/tutoria
 1. `git checkout <branch-name>`
 1. Make changes to the code base and commit them using e.g. `git commit -a -m 'Look ma, I did it'`
 1. When you're done:
- 1. [Squash your commits](http://www.andrewconnell.com/blog/squash-multiple-git-commits-into-one) into one. There are [several ways](http://stackoverflow.com/a/5201642/131929) of doing this.
+ 1. [Squash your commits](http://www.andrewconnell.com/blog/squash-multiple-git-commits-into-one). There are [several ways](http://stackoverflow.com/a/5201642/131929) of doing this.
  1. Bring your fork up-to-date with the NodeMCU upstream repo ([see below](#keeping-your-fork-in-sync)). Then rebase your branch on `dev` running `git rebase dev`.
 1. `git push`
 1. [Create a pull request](https://help.github.com/articles/creating-a-pull-request/) (PR) on GitHub. 
 
-This is just one way of doing things. If you're proficient in Git matters you're free to choose your own. If you want to read more then the [GitHub chapter in the Git book](http://git-scm.com/book/en/v2/GitHub-Contributing-to-a-Project#The-GitHub-Flow) is a way to start. [GitHub's own documenation](https://help.github.com/categories/collaborating/) contains a wealth of information as well.
+This is just one way of doing things. If you're proficient in Git matters you're free to choose your own. If you want to read more then the [GitHub chapter in the Git book](http://git-scm.com/book/en/v2/GitHub-Contributing-to-a-Project#The-GitHub-Flow) is a way to start. [GitHub's own documentation](https://help.github.com/categories/collaborating/) contains a wealth of information as well.
 
 ### Keeping your fork in sync
 You need to sync your fork with the NodeMCU upstream repository from time to time, latest before you rebase (see flow above).

Makefile

@@ -24,6 +24,13 @@ SDK_DIR:=$(TOP_DIR)/sdk/esp_iot_sdk_v$(SDK_VER)
 CCFLAGS:= -I$(TOP_DIR)/sdk-overrides/include -I$(SDK_DIR)/include
 LDFLAGS:= -L$(SDK_DIR)/lib -L$(SDK_DIR)/ld $(LDFLAGS)
 
+ifdef DEBUG
+  CCFLAGS += -ggdb -O0
+  LDFLAGS += -ggdb
+else
+  CCFLAGS += -Os
+endif
+
 #############################################################
 # Select compile
 #
@@ -38,11 +45,11 @@ ifeq ($(OS),Windows_NT)
 		CPP = xt-cpp
 		OBJCOPY = xt-objcopy
 		#MAKE = xt-make
-		CCFLAGS += -Os --rename-section .text=.irom0.text --rename-section .literal=.irom0.literal
+		CCFLAGS += --rename-section .text=.irom0.text --rename-section .literal=.irom0.literal
 	else 
 		# It is gcc, may be cygwin
 		# Can we use -fdata-sections?
-		CCFLAGS += -Os -ffunction-sections -fno-jump-tables -fdata-sections
+		CCFLAGS += -ffunction-sections -fno-jump-tables -fdata-sections
 		AR = xtensa-lx106-elf-ar
 		CC = xtensa-lx106-elf-gcc
 		NM = xtensa-lx106-elf-nm
@@ -69,7 +76,7 @@ else
 	else
 		ESPPORT = $(COMPORT)
 	endif
-	CCFLAGS += -Os -ffunction-sections -fno-jump-tables -fdata-sections
+	CCFLAGS += -ffunction-sections -fno-jump-tables -fdata-sections
 	AR = xtensa-lx106-elf-ar
 	CC = xtensa-lx106-elf-gcc
 	NM = xtensa-lx106-elf-nm
@@ -231,11 +238,22 @@ clobber: $(SPECIAL_CLOBBER)
 	$(foreach d, $(SUBDIRS), $(MAKE) -C $(d) clobber;)
 	$(RM) -r $(ODIR)
 
-flash: 
+flash:
+	@echo "use one of the following targets to flash the firmware"
+	@echo "  make flash512k - for ESP with 512kB flash size"
+	@echo "  make flash4m   - for ESP with   4MB flash size"
+
+flash512k:
+	$(MAKE) -e FLASHOPTIONS="-fm qio -fs  4m -ff 40m" flashinternal
+
+flash4m:
+	$(MAKE) -e FLASHOPTIONS="-fm dio -fs 32m -ff 40m" flashinternal
+
+flashinternal:
 ifndef PDIR
-	$(MAKE) -C ./app flash
+	$(MAKE) -C ./app flashinternal
 else
-	$(ESPTOOL) --port $(ESPPORT) write_flash 0x00000 $(FIRMWAREDIR)0x00000.bin 0x10000 $(FIRMWAREDIR)0x10000.bin
+	$(ESPTOOL) --port $(ESPPORT) write_flash $(FLASHOPTIONS) 0x00000 $(FIRMWAREDIR)0x00000.bin 0x10000 $(FIRMWAREDIR)0x10000.bin
 endif
 
 .subdirs:
@@ -255,7 +273,7 @@ endif
 .PHONY: spiffs-image-remove
 
 spiffs-image-remove:
-	$(MAKE) -C tools remove-image
+	$(MAKE) -C tools remove-image spiffsimg/spiffsimg
 
 .PHONY: spiffs-image
 

app/Makefile

@@ -42,7 +42,10 @@ SUBDIRS= 					\
 	dhtlib					\
 	tsl2561					\
 	net					\
-	http
+	http					\
+	fatfs					\
+	esp-gdbstub				\
+	websocket
 
 endif # } PDIR
 
@@ -82,11 +85,14 @@ COMPONENTS_eagle.app.v6 = 			\
 	ucglib/ucglib.a 			\
 	smart/smart.a 				\
 	spiffs/spiffs.a 			\
+	fatfs/libfatfs.a			\
 	cjson/libcjson.a 			\
 	crypto/libcrypto.a 			\
 	dhtlib/libdhtlib.a 			\
 	tsl2561/tsl2561lib.a			\
 	http/libhttp.a				\
+	websocket/libwebsocket.a		\
+	esp-gdbstub/libgdbstub.a		\
 	net/libnodemcu_net.a			\
 	modules/libmodules.a			\
 

app/cjson/cjson_mem.c

@@ -2,27 +2,23 @@
 #include "../lua/lauxlib.h"
 #include <c_stdlib.h>
 
-static lua_State *gL;
 static const char errfmt[] = "cjson %salloc: out of mem (%d bytes)";
 
-void cjson_mem_setlua (lua_State *L)
-{
-  gL = L;
-}
-
 void *cjson_mem_malloc (uint32_t sz)
 {
   void *p = (void*)c_malloc (sz);
-  if (!p && gL)
-    luaL_error (gL, errfmt, "m", sz);
+  lua_State *L = lua_getstate();
+  if (!p)
+    luaL_error (L, errfmt, "m", sz);
   return p;
 }
 
 
 void *cjson_mem_realloc (void *o, uint32_t sz)
 {
   void *p = (void*)c_realloc (o, sz);
-  if (!p && gL)
-    luaL_error (gL, errfmt, "re", sz);
+  lua_State *L = lua_getstate();
+  if (!p)
+    luaL_error (L, errfmt, "re", sz);
   return p;
 }

app/cjson/cjson_mem.h

@@ -3,8 +3,6 @@
 
 #include "../lua/lua.h"
 
-void cjson_mem_setlua (lua_State *L);
-
 void *cjson_mem_malloc (uint32_t sz);
 void *cjson_mem_realloc (void *p, uint32_t sz);
 

app/coap/coap.h

@@ -155,7 +155,6 @@ typedef struct
 typedef struct coap_luser_entry coap_luser_entry;
 
 struct coap_luser_entry{
-    lua_State *L;
     // int ref;
     // char name[MAX_SEGMENTS_SIZE+1];         // +1 for string '\0'
     const char *name;

app/coap/endpoints.c

@@ -37,6 +37,7 @@ static int handle_get_variable(const coap_endpoint_t *ep, coap_rw_buffer_t *scra
     const coap_option_t *opt;
     uint8_t count;
     int n;
+    lua_State *L = lua_getstate();
     if (NULL != (opt = coap_findOptions(inpkt, COAP_OPTION_URI_PATH, &count)))
     {
         if ((count != ep->path->count ) && (count != ep->path->count + 1)) // +1 for /f/[function], /v/[variable]
@@ -58,19 +59,17 @@ static int handle_get_variable(const coap_endpoint_t *ep, coap_rw_buffer_t *scra
                     NODE_DBG("/v1/v/");
                     NODE_DBG((char *)h->name);
                     NODE_DBG(" match.\n");
-                    if(h->L == NULL)
-                        return coap_make_response(scratch, outpkt, NULL, 0, id_hi, id_lo, &inpkt->tok, COAP_RSPCODE_NOT_FOUND, COAP_CONTENTTYPE_NONE);
                     if(c_strlen(h->name))
                     {
-                        n = lua_gettop(h->L);
-                        lua_getglobal(h->L, h->name);
-                        if (!lua_isnumber(h->L, -1) && !lua_isstring(h->L, -1)) {
+                        n = lua_gettop(L);
+                        lua_getglobal(L, h->name);
+                        if (!lua_isnumber(L, -1) && !lua_isstring(L, -1)) {
                             NODE_DBG ("should be a number or string.\n");
-                            lua_settop(h->L, n);
+                            lua_settop(L, n);
                             return coap_make_response(scratch, outpkt, NULL, 0, id_hi, id_lo, &inpkt->tok, COAP_RSPCODE_NOT_FOUND, COAP_CONTENTTYPE_NONE);
                         } else {
-                            const char *res = lua_tostring(h->L,-1);
-                            lua_settop(h->L, n);
+                            const char *res = lua_tostring(L,-1);
+                            lua_settop(L, n);
                             return coap_make_response(scratch, outpkt, (const uint8_t *)res, c_strlen(res), id_hi, id_lo, &inpkt->tok, COAP_RSPCODE_CONTENT, h->content_type);
                         }
                     }
@@ -94,6 +93,7 @@ static int handle_post_function(const coap_endpoint_t *ep, coap_rw_buffer_t *scr
     const coap_option_t *opt;
     uint8_t count;
     int n;
+    lua_State *L = lua_getstate();
     if (NULL != (opt = coap_findOptions(inpkt, COAP_OPTION_URI_PATH, &count)))
     {
         if ((count != ep->path->count ) && (count != ep->path->count + 1)) // +1 for /f/[function], /v/[variable]
@@ -116,37 +116,34 @@ static int handle_post_function(const coap_endpoint_t *ep, coap_rw_buffer_t *scr
                     NODE_DBG((char *)h->name);
                     NODE_DBG(" match.\n");
 
-                    if(h->L == NULL)
-                        return coap_make_response(scratch, outpkt, NULL, 0, id_hi, id_lo, &inpkt->tok, COAP_RSPCODE_NOT_FOUND, COAP_CONTENTTYPE_NONE);
-
                     if(c_strlen(h->name))
                     {
-                        n = lua_gettop(h->L);
-                        lua_getglobal(h->L, h->name);
-                        if (lua_type(h->L, -1) != LUA_TFUNCTION) {
+                        n = lua_gettop(L);
+                        lua_getglobal(L, h->name);
+                        if (lua_type(L, -1) != LUA_TFUNCTION) {
                             NODE_DBG ("should be a function\n");
-                            lua_settop(h->L, n);
+                            lua_settop(L, n);
                             return coap_make_response(scratch, outpkt, NULL, 0, id_hi, id_lo, &inpkt->tok, COAP_RSPCODE_NOT_FOUND, COAP_CONTENTTYPE_NONE);
                         } else {
-                            lua_pushlstring(h->L, inpkt->payload.p, inpkt->payload.len);     // make sure payload.p is filled with '\0' after payload.len, or use lua_pushlstring
-                            lua_call(h->L, 1, 1);
-                            if (!lua_isnil(h->L, -1)){  /* get return? */
-                                if( lua_isstring(h->L, -1) )   // deal with the return string
+                            lua_pushlstring(L, inpkt->payload.p, inpkt->payload.len);     // make sure payload.p is filled with '\0' after payload.len, or use lua_pushlstring
+                            lua_call(L, 1, 1);
+                            if (!lua_isnil(L, -1)){  /* get return? */
+                                if( lua_isstring(L, -1) )   // deal with the return string
                                 {
                                     size_t len = 0;
-                                    const char *ret = luaL_checklstring( h->L, -1, &len );
+                                    const char *ret = luaL_checklstring( L, -1, &len );
                                     if(len > MAX_PAYLOAD_SIZE){
-                                        lua_settop(h->L, n);
-                                        luaL_error( h->L, "return string:<MAX_PAYLOAD_SIZE" );
+                                        lua_settop(L, n);
+                                        luaL_error( L, "return string:<MAX_PAYLOAD_SIZE" );
                                         return coap_make_response(scratch, outpkt, NULL, 0, id_hi, id_lo, &inpkt->tok, COAP_RSPCODE_NOT_FOUND, COAP_CONTENTTYPE_NONE);
                                     }
                                     NODE_DBG((char *)ret);
                                     NODE_DBG("\n");
-                                    lua_settop(h->L, n);
+                                    lua_settop(L, n);
                                     return coap_make_response(scratch, outpkt, ret, len, id_hi, id_lo, &inpkt->tok, COAP_RSPCODE_CONTENT, COAP_CONTENTTYPE_TEXT_PLAIN);
                                 } 
                             } else {
-                                lua_settop(h->L, n);
+                                lua_settop(L, n);
                                 return coap_make_response(scratch, outpkt, NULL, 0, id_hi, id_lo, &inpkt->tok, COAP_RSPCODE_CONTENT, COAP_CONTENTTYPE_TEXT_PLAIN);
                             }
                         }
@@ -198,10 +195,10 @@ static int handle_get_id(const coap_endpoint_t *ep, coap_rw_buffer_t *scratch, c
     return coap_make_response(scratch, outpkt, (const uint8_t *)(&id), sizeof(uint32_t), id_hi, id_lo, &inpkt->tok, COAP_RSPCODE_CONTENT, COAP_CONTENTTYPE_TEXT_PLAIN);
 }
 
-coap_luser_entry var_head = {NULL,NULL,NULL,0};
+coap_luser_entry var_head = {NULL,NULL,0};
 coap_luser_entry *variable_entry = &var_head;
 
-coap_luser_entry func_head = {NULL,NULL,NULL,0};
+coap_luser_entry func_head = {NULL,NULL,0};
 coap_luser_entry *function_entry = &func_head;
 
 const coap_endpoint_t endpoints[] =

app/crypto/Makefile

@@ -41,6 +41,7 @@ STD_CFLAGS=-std=gnu11 -Wimplicit
 INCLUDES := $(INCLUDES) -I $(PDIR)include
 INCLUDES += -I ./
 INCLUDES += -I ../libc
+INCLUDES += -I ../platform
 PDIR := ../$(PDIR)
 sinclude $(PDIR)Makefile