图片文件如果相同，md5是一样的。于是利用此进行图片去重。





```python
import os
import hashlib
def get_md5(file):
    file = open(file, 'rb')
    md5 = hashlib.md5(file.read())
    file.close()
    md5_values = md5.hexdigest()
    return md5_values
file_path =  r"D:\Users\Administrator\Desktop\smokingsig\yolov5k\images"
os.chdir(file_path)
file_list = os.listdir(file_path)
md5_list = []
for file in file_list:
    md5 = get_md5(file)
    if md5 not in md5_list:
        md5_list.append(md5)
    else:
        os.remove(file)
```

相似度  计算量非常大，不太好用：
```python
# -*- encoding=utf-8 -*-
import cv2
import os
class dup_picture:
    def __init__(self):
        self.orb = cv2.ORB_create()
        self.bf = cv2.BFMatcher(cv2.NORM_HAMMING)
    def get_pic_desc(self, img1_path: str):
        img1 = cv2.imread(img1_path, cv2.IMREAD_GRAYSCALE)
        kp1, des1 = self.orb.detectAndCompute(img1, None)
        return des1
    def get_similary(self, des1, des2):
        # knn筛选结果
        matches = self.bf.knnMatch(des1, trainDescriptors=des2, k=2)
        # 查看最大匹配点数目
        good = [m for (m, n) in matches if m.distance < 0.75 * n.distance]
        if len(matches) == 0:
            return 0
        similary = len(good) / len(matches)
        return similary
if __name__ == '__main__':
    duptool = dup_picture()
    file_path = r"D:\PycharmProjects\downloadpicture\all_smoking_picture"
    file_list = os.listdir(file_path)
    file_list = list(filter(lambda x: x.endswith(".jpg"), file_list))  # must .jpg pic
    os.chdir(file_path)
    unique_list = [duptool.get_pic_desc(file_list[0]), ]
    for k,file in enumerate(file_list[1:]):
        print(k)
        des = duptool.get_pic_desc(file)
        if des is None:
            os.remove(file)
            continue
        for des_i in unique_list:
            if duptool.get_similary(des, des_i) > 0.5:
                os.remove(file)
                print(file)
                break
        else:  # Smooth and uninterrupted execution completed
            unique_list.append(des)
```


其余labels处理：

```python
import os
imagespath = r"D:\Users\Administrator\Desktop\smokingsig\yolov5k\images"
labelspath = r"D:\Users\Administrator\Desktop\yololabels"
img_list = os.listdir(imagespath)
lab_list = os.listdir(labelspath)
# ---- read label ----> if none label ----> delete label file
for name in lab_list:
    lfile = os.path.join(labelspath, name)
    if "image" in name:
        with open(lfile, "r") as f:
            res = f.readlines()
        if len(res) == 0:
            os.remove(lfile)
# ---- read label ----> if  label str too short ----> delete label file
for name in lab_list:
    lfile = os.path.join(labelspath, name)
    if "image" in name:
        with open(lfile, "r") as f:
            res = f.readlines()
        if len(res[0]) < 3:
            os.remove(lfile)
# ---- read images names ----> if have not label file ----> delete imgfile
for name in img_list:
    if name[:-4] + ".txt" not in lab_list:
        os.remove(os.path.join(imagespath, name))
```

Python 图片去重 图片文件去重 依据相似度去重 cv2.ORB

# 四种POST 请求方式：





[https://imququ.com/post/four-ways-to-post-data-in-http.html#toc-2](https://imququ.com/post/four-ways-to-post-data-in-http.html#toc-2)

Content-Type: application/x-www-form-urlencoded  最原始表单，参数单一，不能传文件。

Content-Type:multipart/form-data  改进后的表单，能传文件。

Content-Type: application/json  倾向于传json，json规范导致前后端传数据都喜欢用这个，数据生产和消费简单。

# FastAPI 中如何支持

[https://fastapi.tiangolo.com/zh/tutorial/schema-extra-example/](https://fastapi.tiangolo.com/zh/tutorial/schema-extra-example/)

把函数参数改为Body解析即可，而不是FORM解析。

```python
# -*- coding:utf-8 -*-
import traceback
import uvicorn
from typing import Optional
from fastapi import Body, FastAPI
from pydantic import BaseModel
from fastapi.middleware.cors import CORSMiddleware
app = FastAPI(
    title='FastAPI',
    description='FastAPI',
    version='1.0.0',
    docs_url='/docs',
    redoc_url='/redocs',
)
app.add_middleware(
    CORSMiddleware,
    allow_origins=["*"],
    allow_credentials=True,
    allow_methods=["*"],
    allow_headers=["*"],
)
class Item(BaseModel):
    name: str
    description: Optional[str] = None
    price: float
    tax: Optional[float] = None
@app.post("/items/{item_id}")
async def update_item(
        item_id: int,
        item: Item = Body(
            ...,
            example={
                "name": "Foo",
                "description": "A very nice Item",
                "price": 35.4,
                "tax": 3.2,
            },
        ),
):
    results = {"item_id": item_id, "item": item}
    return results
if __name__ == '__main__':
    uvicorn.run('fastapia:app', host='0.0.0.0', port=8001, reload=False, debug=False, workers=1)
```

# PostMan 测试

修改请求头

![在这里插入图片描述](/static/img/images/00319.png)

写raw的原生json字符串

![在这里插入图片描述](/static/img/images/00320.png)


```python
import requests
requests.post(url="http://0.0.0.0:8001/items/1",
              json={
                  "name": "Foo",
                  "description": "A very nice Item",
                  "price": 35.4,
                  "tax": 3.2,
              },
              )
```

FastAPI python content-type: application/json

参考:[https://blog.csdn.net/haveanybody/article/details/86494063](https://blog.csdn.net/haveanybody/article/details/86494063)





# 基础

cv2 打开图片，tensor的shape是（im_height, im_width, channel），BGR格式。**cv2.imread返回的是numpy.ndarray张量。**

pillow 打开图片，RGB格式。Pillow读出来的是PIL.Image.Image对象。

# 打开图片与保存图片

```python
##PIL 读取、保存图片方法
from PIL import Image
img = Image.open("eee.jpg")
img.save("eee_result.jpg")
##cv2 读取、保存图片方法
import cv2
img = cv2.imread("eee.jpg")
cv2.imwrite("eee_result.jpg", img)
##图片文件打开为base64
import base64
def img_base64(img_path):
    with open(img_path, "rb") as f:
        base64_str = base64.b64encode(f.read())
    return base64_str
```

# CV2中的  BGR转RGB

```python
from matplotlib import pyplot as plt
import cv2
img = cv2.imread("eee.jpg")
cv2.imwrite("eee_result.jpg", img)
imgrgb = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
plt.imshow(imgrgb)
plt.show()
```

# PIL CV2 的相互转换

```python
## PIL转cv2
import cv2
from PIL import Image
import numpy as np
img_rgb = Image.open("eee.jpg")
img_bgr = cv2.cvtColor(np.asarray(img_rgb), cv2.COLOR_RGB2BGR) # 成为cv2的numpy.ndarray
```

```python
## PIL转cv2
import cv2
from PIL import Image
img_bgr = cv2.imread("eee.jpg")
img_rgb = Image.fromarray(cv2.cvtColor(img_bgr, cv2.COLOR_BGR2RGB)) # 成为Pillow的PIL.Image.Image对象
```


# PIL base64 的相互转换

```python
##PIL转base64
import base64
from io import BytesIO
from PIL import Image
def pillow_to_base64(image):
    img_buffer = BytesIO()
    image.save(img_buffer, format='JPEG')
    byte_data = img_buffer.getvalue()
    base64_str = base64.b64encode(byte_data)
    return base64_str
# example
img = Image.open("eee.jpg")
print(pillow_to_base64(img))
```

```python
##PIL转base64
import base64
from io import BytesIO
from PIL import Image
def base64_to_pillow(base64_str):
    image = base64.b64decode(base64_str)
    image = BytesIO(image)
    image = Image.open(image)
    return image
# example
with open("eee.jpg", "rb") as f:
    base64_str = base64.b64encode(f.read())
print(base64_to_pillow(base64_str))
```

# CV2 base64 的相互转换

```python
# cv2 转 base64
import base64
import cv2
def cv2_to_base64(image):
    base64_str = cv2.imencode('.jpg', image)[1].tostring()
    base64_str = base64.b64encode(base64_str)
    return base64_str
# example
image = cv2.imread("eee.jpg")
print(cv2_to_base64(image))
```

```python
# base64 转 cv2
import base64
import numpy as np
import cv2
from PIL import Image
from io import BytesIO
def base64_to_cv2(base64_str):
    imgString = base64.b64decode(base64_str)
    nparr = np.frombuffer(imgString, np.uint8)
    image = cv2.imdecode(nparr, cv2.IMREAD_COLOR)
    return image
# example
with open("eee.jpg", "rb") as f:
    base64_str = base64.b64encode(f.read())
print(base64_to_cv2(base64_str))
```


# FastAPI中bytes 转CV2

```python
img = cv2.imdecode(np.frombuffer(file, dtype=np.uint8), cv2.IMREAD_COLOR)
```

# FastAPI中uploadfile  转CV2

```python
img = cv2.imdecode(np.fromstring(fileb.file.read(), np.uint8), cv2.IMREAD_COLOR)
```


# CV2 numpy  转成 字节流（open("file.jpg",'rb')）

参考：[https://blog.csdn.net/sinat_29957455/article/details/107208432](https://blog.csdn.net/sinat_29957455/article/details/107208432)

```python
img = cv2.imread("lpr1.png")
# 对数组的图片格式进行编码
success, encoded_image = cv2.imencode(".jpg", img)
# 将数组转为bytes
f = encoded_image.tobytes()
```

cv2 pillow base64 bytes uploadfile 图片文件之间的转换 FastAPI

#  FastAPI 服务





下面是代码，

`@app.post("/files/")` 路由让我们可以传文件。

`@app.post("/uploadfiles/")`路由让我们可以传多个文件。

需要保证：

（1） uvicorn.run中写的是文件名：app名，确保自己的文件名。

（2）`# python 3.8   pip install fastapi uvicorn python-multipart`

```python
# -*- coding:utf-8 -*-
import traceback
from typing import List, Optional, Union
import uvicorn
from fastapi.middleware.cors import CORSMiddleware
from fastapi import FastAPI, File, Form, UploadFile
import cv2
import numpy as np
app = FastAPI(
    title='FastAPI Tutorial',
    description='FastAPI教程',
    version='1.0.0',
    docs_url='/docs',
    redoc_url='/redocs',
)
app.add_middleware(
    CORSMiddleware,
    allow_origins=["*"],
    allow_credentials=True,
    allow_methods=["*"],
    allow_headers=["*"],
)
@app.post("/files/")
async def alg_file(
        fileb: UploadFile = File(...),
        token: str = Form(...)
):
    try:
        img = cv2.imdecode(np.fromstring(fileb.file.read(), np.uint8), cv2.IMREAD_COLOR)
        return {
            "token": token,
            "filename": fileb.filename,
            "content_type": fileb.content_type,
            "size": img.shape
        }
    except:
        traceback.print_exc()
        return -1
@app.post("/uploadfiles/")
async def create_upload_files(files: List[UploadFile] = File(...)):
    return {"filenames": [file.filename for file in files]}
@app.post("/filesper/")
def filesper(
        file: Optional[bytes] = File(None),
        fileb: Optional[UploadFile] = File(None),
        filepath: Optional[str] = Form(None),
):
    try:
        if file:
            img = cv2.imdecode(np.frombuffer(file, dtype=np.uint8), cv2.IMREAD_COLOR)
            return {
                "size": img.shape,
                "style": "bytes"
            }
        if fileb:
            img = cv2.imdecode(np.fromstring(fileb.file.read(), np.uint8), cv2.IMREAD_COLOR)
            return {
                "filename": fileb.filename,
                "content_type": fileb.content_type,
                "size": img.shape,
                "style": "UploadFile"
            }
        if filepath:
            return {
                "str": filepath,
                "style": "str"
            }
    except:
        traceback.print_exc()
        return -1
if __name__ == '__main__':
    uvicorn.run('yolo5fastapi:app', host='0.0.0.0', port=8001, reload=False, debug=False, workers=1)
```


#  requests 测试

## requests 单一文件上传

测试上传一个图片文件：

```python
import requests
url = "http://0.0.0.0:8001/files/"
with open("/home/kevin_xie/yifeinfs/pycharm_k/yolov5_v2/kevin/picture/realcamera.jpg", "rb") as f:
    res = requests.post(url=url, data={"token": "sdas"}, files={"fileb": f}, timeout=None)
    print(res.json())
```

返回：

```python
{'token': 'sdas', 'filename': 'realcamera.jpg', 'content_type': '', 'size': [720, 1280, 3]}
```

## requests 多个文件上传

注意给文件参数的方式，元组和列表均可，可自由尝试。
```python
import requests
url = "http://0.0.0.0:8001/uploadfiles/"
with open("/home/kevin_xie/yifeinfs/pycharm_k/yolov5_v2/kevin/picture/realcamera.jpg", "rb") as f:
    res = requests.post(url=url, files=[("files", f), ("files", f), ("files", f)], timeout=None)
    print(res.json())
```

Python fastapi requests 传文件方式 上传文件 上传多个文件 UploadFile模板方式 上传图片

# 1 Flask 路由阻塞





当启动一个Flask APP如下，在同一浏览器开2个窗口访问这个服务（我的是http://172.18.43.32:8001/hello），前一个请求没返回时，另外一个窗口的相同请求不会有反应。

这是浏览器的锅，同一请求会顺序执行，此时可以用同一台电脑的不同浏览器测试，可以发现不会存在路由阻塞。
```python
import time
from flask import Flask
app = Flask(__name__)
@app.route('/hello')
def hello_world():
    print("receive !")
    time.sleep(20)
    return 'Hello World!'
if __name__ == '__main__':
    app.run(host='0.0.0.0', port=8001)
```

# 2 requests访问

- 换个方式，使用requests访问，线程池中100个线程同时执行访问：

```python
import requests
from concurrent.futures import ThreadPoolExecutor
def task():
    res = requests.request(method="GET", url="http://172.18.43.32:8001/hello", timeout=None)
executor = ThreadPoolExecutor(max_workers=100)
for i in range(100):
    executor.submit(task)
```

还是可以看见不会阻塞。


# 3 FastAPI 路由阻塞

下面的情况下，访问就会阻塞。


```python
# -*- coding:utf-8 -*-
import asyncio
import time
import uvicorn
from fastapi.middleware.cors import CORSMiddleware
from fastapi import FastAPI, File, Form, UploadFile
import cv2
import numpy as np
app = FastAPI(
    title='FastAPI Tutorial',
    description='FastAPI教程',
    version='1.0.0',
    docs_url='/docs',
    redoc_url='/redocs',
)
@app.get("/hello")
async def hl():
    print("receive !")
    time.sleep(20)
    return {
        "token": "ads"
    }
def fastweb():
    uvicorn.run('deal:app', host='0.0.0.0', port=8001, reload=False, debug=False, workers=1)
if __name__ == '__main__':
    fastweb()
```

原因：在async标识的函数中使用同步方法time.sleep(20)，这导致了一个异步程序之间的同步问题，出现了错误！这种写法本身就是错误的，加async 是想利用Python的协程机制 将路由变成异步请求， 进一步提高并发量！只要考虑加async ，那么就得好好设计函数内部， 多用一些异步方法，比如可以写成：

```python
@app.get("/hello")
async def hl():
    print("receive !")
    await asyncio.sleep(20)
    return {
        "token": "ads"
    }
```

设计合理就会极大利用起协程异步，从而极大提高并发量，设计不合理就是呆瓜。若是分不清，还是老老实实和Flask一样写法：

```python
@app.get("/hello")
def hl():
    print("receive !")
    time.sleep(20)
    return {
        "token": "ads"
    }
```

这样也是出现不了路由阻塞的。

# 4 总结：同步并发 异步并发

1 Flask 中的属于同步并发，可以无阻塞访问！但要注意别让浏览器欺骗你！

2 FastAPI 支持异步框架，支持是支持，前提是你得用好它，不然就呆瓜。用好了并发量就提升了！

3 FastAPI 中同样支持Flask 那样子的同步并发，和Flask 一样一样的，但要注意写法！

4 FastAPI 貌似比 Flask 更强大！

# 参考

java web

golang 协程

Flask FastAPI 路由阻塞 Web Python 访问同一url出现阻塞 同步 异步 并发

# 何为bytes？





是Python3的数据类型，详见：[http://c.biancheng.net/view/2175.html](http://c.biancheng.net/view/2175.html)。

# np.frombuffer

返回一维数组：

![在这里插入图片描述](/static/img/images/00321.png)

# cv2.imdecode

cv2.imdecode()函数从指定的内存缓存中读取数据，并把数据转换(解码)成图像格式;主要用于从网络传输数据中恢复出图像。

cv2.imencode()函数是将图片格式转换(编码)成流数据，赋值到内存缓存中;主要用于图像数据格式的压缩，方便网络传输。

# bytes转图片numpy矩阵
```python
img = cv2.imdecode(np.frombuffer(file, dtype=np.uint8), cv2.IMREAD_COLOR)
```

# 逆运算 numpy矩阵转bytes

这里的img_bytes 就是bytes的file
```python
# 对数组的图片格式进行编码
success, encoded_image = cv2.imencode(".jpg", img)
# 将数组转为bytes
img_bytes = encoded_image.tostring()
```

# bytes转图片numpy矩阵


# 完整代码：

```python
# -*- coding:utf-8 -*-
import uvicorn
from fastapi.middleware.cors import CORSMiddleware
from fastapi import FastAPI, File, Form, UploadFile
import cv2
import numpy as np
from io import BytesIO
app = FastAPI(
    title='FastAPI Tutorial',
    description='FastAPI教程',
    version='1.0.0',
    docs_url='/docs',
    redoc_url='/redocs',
)
app.add_middleware(
    CORSMiddleware,
    allow_origins=["*"],
    allow_credentials=True,
    allow_methods=["*"],
    allow_headers=["*"],
)
@app.post("/files/")
async def alg_file(
        file: bytes = File(...),
        token: str = Form(...)
):
    img = cv2.imdecode(np.frombuffer(file, dtype=np.uint8), 1)
    return {
        "token": token,
        "size": img.shape
    }
if __name__ == '__main__':
    uvicorn.run('run1:app', host='0.0.0.0', port=8002, reload=True, debug=True, workers=1)
```

FastAPI bytes转矩阵 bytes转图片 bytes opencv URL传文件 POST FORM Python

Opencv 读取的图片是BGR 矩阵：




```python
import cv2 
import matplotlib.pyplot as plt
img=cv2.imread('./res.jpg')
plt.imshow(img)
plt.show()
```

plt显示方法imshow是默认矩阵是RGB的，

![在这里插入图片描述](/static/img/images/00322.png)

想要正确显示，需要：

```python
imgrgb= cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
plt.imshow(imgrgb)
plt.show()
```

![在这里插入图片描述](/static/img/images/00323.png)

Opencv Python 图片通道 BGR 转 RGB imgrgb= cv2.cvtColor(img, cv2.COLOR_BGR2RGB)

读取




```python
import cv2
import numpy as np 
img = cv2.imdecode(np.fromfile("./人物-1.jpg", dtype=np.uint8), 1) # img是矩阵
```

写入：

```python
import cv2
cv2.imencode('.jpg', img)[1].tofile(os.path.join(self.keyword, self.keyword + str(count) + ".jpg"))
```

Python cv2.imread 无法打开中文路径的图像

当有字典x和字典y，想合并为字典z。





-------------------------------

Python 3.9 + 提供了最简单的方法：

`z = x | y          # NOTE: 3.9+ ONLY`

-------------------------------

Python 3.5 + 的方法：

`z = {**x, **y}`

-------------------------------

Python 3.4 or lower  的方法，使用update方法写一个合并方法：

```python
def merge_dicts(*dict_args):
    """
    Given any number of dictionaries, shallow copy and merge into a new dict,
    precedence goes to key-value pairs in latter dictionaries.
    """
    result = {}
    for dictionary in dict_args:
        result.update(dictionary)
    return result
```

或者直接：

`z = x.update( y )`

-------------------------------

参考1 ： [stackoverflow](https://stackoverflow.com/questions/38987/how-do-i-merge-two-dictionaries-in-a-single-expression-taking-union-of-dictiona)

Python 字典合并 几种方式 优雅的字典合并 如何优雅地合并python字典

是文件，不会获取路径。





```python
def listPathAllfiles( dirname):
    result = []
    for maindir, subdir, file_name_list in os.walk(dirname):
        for filename in file_name_list:
            apath = os.path.join(maindir, filename)
            result.append(apath)
    return result
```