python中的Dropshipping研究工具演示-DEV365 开发者社区

Intro
App Demo
Virtual Environment and Libraries Installation
Full Code
Code Explanation
Links

Intro

创建此博客文章是为了向您展示如何将Serpapi集成到这个小应用程序中，并解释了该应用程序如何运行和Serpapi在其中的角色。

主要想法来自一个名为Zik Analytics的工具，该工具由Dropshippers使用。

此演示显示了沃尔玛和eBay产品之间的比较，以确定利润。用户选择产品和销售场所，该程序在另一个地方找到相同的产品并计算利润。

app demo

ð注意：使用深色精简主题正常查看表。

在

如果要使用自己的API键，请按照以下步骤：

克隆存储库：

$ git clone https://github.com/chukhraiartur/dropshipping-tool-demo.git

安装依赖项：

$ cd dropshipping-tool-demo && pip install -r requriements.txt

添加当前外壳的SerpApi api key，所有过程从当前的外壳开始：

# used to parse Walmart and eBay results, has a plan of 100 free searches
$ export SERPAPI_API_KEY=<your-api-key>

运行应用程序：

$ streamlit run main.py

完整代码

from serpapi import EbaySearch, WalmartSearch
import streamlit as st
import streamlit.components.v1 as components
import pandas as pd
import time, os, Levenshtein


def get_walmart_results(query: str):
    params = {
        'api_key': os.getenv('SERPAPI_API_KEY'),    # https://serpapi.com/manage-api-key
        'engine': 'walmart',                        # search engine
        'query': query,                             # search query
    }

    search = WalmartSearch(params)                  # data extraction on the SerpApi backend
    results = search.get_dict()                     # JSON -> Python dict

    return results.get('organic_results', [])


def get_ebay_results(query: str):
    params = {
        'api_key': os.getenv('SERPAPI_API_KEY'),    # https://serpapi.com/manage-api-key
        'engine': 'ebay',                           # search engine
        '_nkw': query,                              # search query
        'ebay_domain': 'ebay.com',                  # ebay domain
    }

    search = EbaySearch(params)                     # data extraction on the SerpApi backend
    results = search.get_dict()                     # JSON -> Python dict

    return results.get('organic_results', [])


def compare_walmart_with_ebay(query: str, number_of_products: int, percentage_of_uniqueness: float):
    data = []

    walmart_results = get_walmart_results(query)

    for walmart_result in walmart_results[:number_of_products]:
        ebay_results = get_ebay_results(walmart_result['title'])

        for ebay_result in ebay_results:
            if Levenshtein.ratio(walmart_result['title'], ebay_result['title']) < percentage_of_uniqueness:
                continue

            walmart_price = walmart_result.get('primary_offer', {}).get('offer_price')
            ebay_price = ebay_result.get('price', {}).get('extracted')

            if not ebay_price:
                ebay_price = ebay_result.get('price', {}).get('from', {}).get('extracted')

            profit = 0

            if walmart_price and ebay_price:
                profit = round(walmart_price - ebay_price, 2)

            data.append({
                'Walmart': {
                    'thumbnail': walmart_result['thumbnail'],
                    'title': walmart_result['title'],
                    'link': walmart_result['product_page_url'],
                    'price': walmart_price
                },
                'eBay': {
                    'thumbnail': ebay_result['thumbnail'],
                    'title': ebay_result['title'],
                    'link': ebay_result['link'],
                    'price': ebay_price
                },
                'Profit': profit
            })

    return data


def compare_ebay_with_walmart(query: str, number_of_products: int, percentage_of_uniqueness: float):
    data = []

    ebay_results = get_ebay_results(query)

    for ebay_result in ebay_results[:number_of_products]:
        walmart_results = get_walmart_results(ebay_result['title'])

        for walmart_result in walmart_results:
            if Levenshtein.ratio(ebay_result['title'], walmart_result['title']) < percentage_of_uniqueness:
                continue

            ebay_price = ebay_result.get('price', {}).get('extracted')
            walmart_price = walmart_result.get('primary_offer', {}).get('offer_price')

            if not ebay_price:
                ebay_price = ebay_result.get('price', {}).get('from', {}).get('extracted')

            profit = 0

            if ebay_price and walmart_price:
                profit = round(ebay_price - walmart_price, 2)

            data.append({
                'eBay': {
                    'thumbnail': ebay_result['thumbnail'],
                    'title': ebay_result['title'],
                    'link': ebay_result['link'],
                    'price': ebay_price
                },
                'Walmart': {
                    'thumbnail': walmart_result['thumbnail'],
                    'title': walmart_result['title'],
                    'link': walmart_result['product_page_url'],
                    'price': walmart_price
                },
                'Profit': profit
            })

    return data


def create_table(data: list, where_to_sell: str):
    with open('table_style.css') as file:
        style = file.read()

    products = ''

    for product in data:
        profit_color = 'lime' if product['Profit'] >= 0 else 'red'

        if where_to_sell == 'Walmart':
            products += f'''
            <tr>
                <td><div><img src="{product['Walmart']['thumbnail']}" width="50"></div></td>
                <td><div><a href="{product['Walmart']['link']}" target="_blank">{product['Walmart']['title']}</div></td>
                <td><div>{str(product['Walmart']['price'])}$</div></td>
                <td><div><img src="{product['eBay']['thumbnail']}" width="50"></div></td>
                <td><div><a href="{product['eBay']['link']}" target="_blank">{product['eBay']['title']}</div></td>
                <td><div>{str(product['eBay']['price'])}$</div></td>
                <td><div style="color:{profit_color}">{str(product['Profit'])}$</div></td>
            </tr>
            '''
        elif where_to_sell == 'eBay':
            products += f'''
            <tr>
                <td><div><img src="{product['eBay']['thumbnail']}" width="50"></div></td>
                <td><div><a href="{product['eBay']['link']}" target="_blank">{product['eBay']['title']}</div></td>
                <td><div>{str(product['eBay']['price'])}$</div></td>
                <td><div><img src="{product['Walmart']['thumbnail']}" width="50"></div></td>
                <td><div><a href="{product['Walmart']['link']}" target="_blank">{product['Walmart']['title']}</div></td>
                <td><div>{str(product['Walmart']['price'])}$</div></td>
                <td><div style="color:{profit_color}">{str(product['Profit'])}$</div></td>
            </tr>
            '''

    table = f'''
    <style>
        {style}
    </style>
    <table border="1">
        <thead>
            <tr>
                <th colspan="3"><div>{list(data[0].keys())[0]}</div></th>
                <th colspan="3"><div>{list(data[0].keys())[1]}</div></th>
                <th><div>{list(data[0].keys())[2]}</div></th>
            </tr>
        </thead>
        <tbody>{products}</tbody>
    </table>
    '''

    return table


def save_to_json(data: list):
    json_file = pd.DataFrame(data=data).to_json(index=False, orient='table')

    st.download_button(
        label='Download JSON',
        file_name='comparison-results.json',
        mime='application/json',
        data=json_file,
    )


def save_to_csv(data: list):
    csv_file = pd.DataFrame(data=data).to_csv(index=False)

    st.download_button(
        label="Download CSV",
        file_name='comparison-results.csv',
        mime='text/csv',
        data=csv_file
    )


def main():
    st.title('💸Product Comparison')
    st.markdown(body='This demo compares products from Walmart and eBay to find a profit. SerpApi Demo Project ([repository](https://github.com/chukhraiartur/dropshipping-tool-demo)). Made with [Streamlit](https://streamlit.io/) and [SerpApi](http://serpapi.com/) 🧡')

    if 'visibility' not in st.session_state:
        st.session_state.visibility = 'visible'
        st.session_state.disabled = False

    SEARCH_QUERY: str = st.text_input(
        label='Search query',
        placeholder='Search',
        help='Multiple search queries is not supported.'
    )
    WHERE_TO_SELL = st.selectbox(
        label='Where to sell',
        options=('Walmart', 'eBay'),
        help='Select the platform where you want to sell products. The program will look for the same products on another site and calculate the profit.'
    )
    NUMBER_OF_PRODUCTS: int = st.slider(
        label='Number of products to search',
        min_value=1,
        max_value=20,
        value=10,
        help='Limit the number of products to analyze.'
    )
    PERCENTAGE_OF_UNIQUENESS: int = st.slider(
        label='Percentage of uniqueness',
        min_value=1,
        max_value=100,
        value=50,
        help='The percentage of uniqueness is used to compare how similar one title is to another. The higher this parameter, the more accurate the result.'
    )
    SAVE_OPTION = st.selectbox(
        label='Choose file format to save',
        options=(None, 'JSON', 'CSV'),
        help='By default data won\'t be saved. Choose JSON or CSV format if you want to save the results.'
    )

    col1, col2, col3, col4, col5 = st.columns(5)

    with col3:
        submit_button_holder = st.empty()
        submit_search = submit_button_holder.button(label='Compare products')

    if submit_search and not SEARCH_QUERY:
        st.error(body='Looks like you click a button without a search query. Please enter a search query 👆')
        st.stop()

    if submit_search and SEARCH_QUERY and WHERE_TO_SELL:
        with st.spinner(text='Parsing Product Data...'):
            comparison_results = []

            if WHERE_TO_SELL == 'Walmart':
                comparison_results = compare_walmart_with_ebay(SEARCH_QUERY, NUMBER_OF_PRODUCTS, PERCENTAGE_OF_UNIQUENESS/100)
            elif WHERE_TO_SELL == 'eBay':
                comparison_results = compare_ebay_with_walmart(SEARCH_QUERY, NUMBER_OF_PRODUCTS, PERCENTAGE_OF_UNIQUENESS/100)

        parsing_is_success = st.success('Done parsing 🎉')
        time.sleep(1)
        parsing_is_success.empty()
        submit_button_holder.empty()

        comparison_results_header = st.markdown(body='#### Comparison results')

        if comparison_results:
            table = create_table(comparison_results, WHERE_TO_SELL)
            components.html(table, height=len(comparison_results)*62 + 40)
            time.sleep(1)

        with col3:
            start_over_button_holder = st.empty()
            start_over_button = st.button(label='Start over')  # centered button

        if SAVE_OPTION and comparison_results:
            with st.spinner(text=f'Saving data to {SAVE_OPTION}...'):
                if SAVE_OPTION == 'JSON':
                    save_to_json(comparison_results)
                elif SAVE_OPTION == 'CSV':
                    save_to_csv(comparison_results)

            saving_is_success = st.success('Done saving 🎉')

            time.sleep(1)
            saving_is_success.empty()
            submit_button_holder.empty()

            start_over_info_holder = st.empty()
            start_over_info_holder.error(body='To rerun the script, click on the "Start over" button, or refresh the page.')

            if start_over_button:
                comparison_results_header.empty()
                start_over_button_holder.empty()
                start_over_info_holder.empty()

        if SAVE_OPTION and not comparison_results:
            comparison_results_header.empty()

            no_data_holder = st.empty()
            no_data_holder.error(body='No product found. Click "Start Over" button and try different search query.')

            if start_over_button:
                no_data_holder.empty()
                start_over_button_holder.empty()

        if SAVE_OPTION is None and comparison_results:
            start_over_info_holder = st.empty()
            start_over_info_holder.error(body='To rerun the script, click on the "Start over" button, or refresh the page.')

            if start_over_button:
                comparison_results_header.empty()
                start_over_button_holder.empty()
                start_over_info_holder.empty()

        if SAVE_OPTION is None and not comparison_results:
            comparison_results_header.empty()

            no_data_holder = st.empty()
            no_data_holder.error(body='No product found. Click "Start Over" button and try different search query.')

            if start_over_button:
                comparison_results_header.empty()
                no_data_holder.empty()
                start_over_button_holder.empty()


if __name__ == '__main__':
    main()

代码说明

首先，让我们看一下算法的工作原理。该算法中的项目是将在各个标题中描述的函数：

导入库：

from serpapi import EbaySearch, WalmartSearch
import streamlit as st
import streamlit.components.v1 as components
import pandas as pd
import time, os, Levenshtein

图书馆	目的
koude0	Serpapi的Python API包装器，该包装器从15个以上的搜索引擎中解析数据。
koude1	创建美丽的Web应用程序。
koude2	默认情况下默认情况下找到复杂或不可用的组件。
koude3	将数据转换为文件格式。
koude4	在Python中与时间合作。
koude5	读取秘密环境变量。在这种情况下，它是serpapi api键。
koude6	快速计算字符串相似性。

一堆导入后，我们定义了一个main函数，其中一切都会发生。在此功能的开头，添加了应用程序的标题和描述：

def main():
    st.title('💸Product Comparison')
    st.markdown(body='This demo compares products from Walmart and eBay to find a profit. SerpApi Demo Project ([repository](https://github.com/chukhraiartur/dropshipping-tool-demo)). Made with [Streamlit](https://streamlit.io/) and [SerpApi](http://serpapi.com/) 🧡')

接下来是定义koude1 session state。我用它隐藏或解开某些小部件：

if 'visibility' not in st.session_state:
    st.session_state.visibility = 'visible'
    st.session_state.disabled = False

之后，我定义了一个输入字段，两个滑块和两个选择框：

SEARCH_QUERY: str = st.text_input(
    label='Search query',
    placeholder='Search',
    help='Multiple search queries is not supported.'
)
WHERE_TO_SELL = st.selectbox(
    label='Where to sell',
    options=('Walmart', 'eBay'),
    help='Select the platform where you want to sell products. The program will look for the same products on another site and calculate the profit.'
)
NUMBER_OF_PRODUCTS: int = st.slider(
    label='Number of products to search',
    min_value=1,
    max_value=20,
    value=10,
    help='Limit the number of products to analyze.'
)
PERCENTAGE_OF_UNIQUENESS: int = st.slider(
    label='Percentage of uniqueness',
    min_value=1,
    max_value=100,
    value=50,
    help='The percentage of uniqueness is used to compare how similar one title is to another. The higher this parameter, the more accurate the result.'
)
SAVE_OPTION = st.selectbox(
    label='Choose file format to save',
    options=(None, 'JSON', 'CSV'),
    help='By default data won\'t be saved. Choose JSON or CSV format if you want to save the results.'
)

在这里，我正在创建一个中心按钮：

col1, col2, col3, col4, col5 = st.columns(5)

with col3:
    submit_button_holder = st.empty()
    submit_search = submit_button_holder.button(label='Compare products')

if submit_search and not SEARCH_QUERY:
    st.error(body='Looks like you click a button without a search query. Please enter a search query 👆')
    st.stop()

submit_button_holder用于隐藏或解开小部件。
如果用户未提供任何搜索查询，则使用st.stop停止脚本。

如果用户输入搜索查询，则可以销售并单击“比较产品”按钮，则启动了用于比较产品的功能。根据销售地点，触发了不同的功能，但它们是相似的：

if submit_search and SEARCH_QUERY and WHERE_TO_SELL:
    with st.spinner(text='Parsing Product Data...'):
        comparison_results = []

        if WHERE_TO_SELL == 'Walmart':
            comparison_results = compare_walmart_with_ebay(SEARCH_QUERY, NUMBER_OF_PRODUCTS, PERCENTAGE_OF_UNIQUENESS/100)
        elif WHERE_TO_SELL == 'eBay':
            comparison_results = compare_ebay_with_walmart(SEARCH_QUERY, NUMBER_OF_PRODUCTS, PERCENTAGE_OF_UNIQUENESS/100)

    parsing_is_success = st.success('Done parsing 🎉')
    time.sleep(1)
    parsing_is_success.empty()
    submit_button_holder.empty()

如果找到匹配，则必须显示结果。标准简化表不适合我的目的，因此我正在创建一个自定义表：

comparison_results_header = st.markdown(body='#### Comparison results')

if comparison_results:
    table = create_table(comparison_results, WHERE_TO_SELL)

要显示表，我使用了koude11 method。除了传递的表外，此方法还接受渲染内容的高度。表高度值取决于发现的匹配数乘以每个表行len(comparison_results)*62的高度加上表标头高度值40。因此，整个桌子成功显示：

components.html(table, height=len(comparison_results)*62 + 40)

显示数据后，“比较产品”按钮更改为“启动”按钮：

with col3:
    start_over_button_holder = st.empty()
    start_over_button = st.button(label='Start over')

然后，我检查了保存选项和比较结果的存在：

if SAVE_OPTION and comparison_results:
    with st.spinner(text=f'Saving data to {SAVE_OPTION}...'):
        if SAVE_OPTION == 'JSON':
            save_to_json(comparison_results)
        elif SAVE_OPTION == 'CSV':
            save_to_csv(comparison_results)

    saving_is_success = st.success('Done saving 🎉')

    time.sleep(1)
    saving_is_success.empty()
    submit_button_holder.empty()

    start_over_info_holder = st.empty()
    start_over_info_holder.error(body='To rerun the script, click on the "Start over" button, or refresh the page.')

    if start_over_button:
        comparison_results_header.empty()
        start_over_button_holder.empty()
        start_over_info_holder.empty()

if SAVE_OPTION and not comparison_results:
    comparison_results_header.empty()

    no_data_holder = st.empty()
    no_data_holder.error(body='No product found. Click "Start Over" button and try different search query.')

    if start_over_button:
        no_data_holder.empty()
        start_over_button_holder.empty()

if SAVE_OPTION is None and comparison_results:
    start_over_info_holder = st.empty()
    start_over_info_holder.error(body='To rerun the script, click on the "Start over" button, or refresh the page.')

    if start_over_button:
        comparison_results_header.empty()
        start_over_button_holder.empty()
        start_over_info_holder.empty()

if SAVE_OPTION is None and not comparison_results:
    comparison_results_header.empty()

    no_data_holder = st.empty()
    no_data_holder.error(body='No product found. Click "Start Over" button and try different search query.')

    if start_over_button:
        comparison_results_header.empty()
        no_data_holder.empty()
        start_over_button_holder.empty()

您可能已经注意到，在每次检查中，都有对“启动”按钮的其他检查。这是删除不必要的信息以进行新的比较所必需的。

最后，我添加了一个if __name__ == '__main__'成语，该习惯保护用户不打算在不打算时意外调用脚本，并调用将运行整个脚本的main函数：

if __name__ == '__main__':
    main()

从网站获得结果

在此应用程序中，从沃尔玛和eBay检索数据。我将分析如何从沃尔玛提取数据的功能。从eBay提取数据的功能以类似的方式工作。

定义用于生成URL的参数：

def get_walmart_results(query: str):
    params = {
        'api_key': os.getenv('SERPAPI_API_KEY'),    # https://serpapi.com/manage-api-key
        'engine': 'walmart',                        # search engine
        'query': query,                             # search query
    }

参数	解释
koude16	参数定义要使用的Serpapi私钥。您可以在your account -> API key下找到它
koude17	将参数设置为`walmart`使用沃尔玛API引擎。
koude19	参数定义搜索查询。您可以在常规的沃尔玛搜索中使用任何要使用的东西。

然后，我们创建一个search对象，从SERPAPI后端检索数据。在results字典中，我们从JSON获取数据：

search = WalmartSearch(params)      # data extraction on the SerpApi backend
results = search.get_dict()         # JSON -> Python dict

在功能结束时，您需要返回有关产品的信息。如果找不到这样的产品，则返回一个空列表：

return results.get('organic_results', [])

比较产品

如果用户选择沃尔玛作为出售场所，则启动了将沃尔玛与eBay进行比较的功能。也可以反向工作。

在函数开头，声明了将添加数据的列表：

def compare_walmart_with_ebay(query: str, number_of_products: int, percentage_of_uniqueness: float):
    data = []

对于搜索查询，我们从沃尔玛获得了结果。之后，对于沃尔玛的每个产品名称，我们从eBay获得结果：

walmart_results = get_walmart_results(query)

for walmart_result in walmart_results[:number_of_products]:
    ebay_results = get_ebay_results(walmart_result['title'])

将沃尔玛的每个项目与eBay上的每场比赛进行比较。商品的比较通过标题实施。要获得一个类似于另一个字符串的字符串，我使用了koude22 method。如果标题中文本匹配的百分比少于唯一性的百分比，请转到下一个产品：

for ebay_result in ebay_results:
    if Levenshtein.ratio(walmart_result['title'], ebay_result['title']) < percentage_of_uniqueness:
        continue

如果找到了比赛，则检索了该项目的沃尔玛和eBay价格。然后计算利润：

walmart_price = walmart_result.get('primary_offer', {}).get('offer_price')
ebay_price = ebay_result.get('price', {}).get('extracted')

if not ebay_price:
    ebay_price = ebay_result.get('price', {}).get('from', {}).get('extracted')

profit = 0

if walmart_price and ebay_price:
    profit = round(walmart_price - ebay_price, 2)

有关发现匹配的数据已添加到列表中：

data.append({
    'Walmart': {
        'thumbnail': walmart_result['thumbnail'],
        'title': walmart_result['title'],
        'link': walmart_result['product_page_url'],
        'price': walmart_price
    },
    'eBay': {
        'thumbnail': ebay_result['thumbnail'],
        'title': ebay_result['title'],
        'link': ebay_result['link'],
        'price': ebay_price
    },
    'Profit': profit
})

在功能末尾，返回了所有匹配的列表：

return data

创建自定义表

此功能基于传递的数据生成表。

在功能开头，包括表的样式表：

def create_table(data: list, where_to_sell: str):
    with open('table_style.css') as file:
        style = file.read()

接下来，形成了表的每一行。第一列将包含有关用户选择的销售地点的数据。此外，利润中文本的颜色将根据价值而变化：

products = ''

for product in data:
    profit_color = 'lime' if product['Profit'] >= 0 else 'red'

    if where_to_sell == 'Walmart':
        products += f'''
        <tr>
            <td><div><img src="{product['Walmart']['thumbnail']}" width="50"></div></td>
            <td><div><a href="{product['Walmart']['link']}" target="_blank">{product['Walmart']['title']}</div></td>
            <td><div>{str(product['Walmart']['price'])}$</div></td>
            <td><div><img src="{product['eBay']['thumbnail']}" width="50"></div></td>
            <td><div><a href="{product['eBay']['link']}" target="_blank">{product['eBay']['title']}</div></td>
            <td><div>{str(product['eBay']['price'])}$</div></td>
            <td><div style="color:{profit_color}">{str(product['Profit'])}$</div></td>
        </tr>
        '''
    elif where_to_sell == 'eBay':
        products += f'''
        <tr>
            <td><div><img src="{product['eBay']['thumbnail']}" width="50"></div></td>
            <td><div><a href="{product['eBay']['link']}" target="_blank">{product['eBay']['title']}</div></td>
            <td><div>{str(product['eBay']['price'])}$</div></td>
            <td><div><img src="{product['Walmart']['thumbnail']}" width="50"></div></td>
            <td><div><a href="{product['Walmart']['link']}" target="_blank">{product['Walmart']['title']}</div></td>
            <td><div>{str(product['Walmart']['price'])}$</div></td>
            <td><div style="color:{profit_color}">{str(product['Profit'])}$</div></td>
        </tr>
        '''

现在形成了表结构，之后由此函数返回：

table = f'''
<style>
    {style}
</style>
<table border="1">
    <thead>
        <tr>
            <th colspan="3"><div>{list(data[0].keys())[0]}</div></th>
            <th colspan="3"><div>{list(data[0].keys())[1]}</div></th>
            <th><div>{list(data[0].keys())[2]}</div></th>
        </tr>
    </thead>
    <tbody>{products}</tbody>
</table>
'''

return table

在

此功能将匹配列表转换为对应于JSON或CSV格式的数据。

要将数据转换为JSON格式，您需要根据传递的数据创建一个数据帧对象，然后调用koude23 method。 koude24 method用于创建下载按钮：

def save_to_json(data: list):
    json_file = pd.DataFrame(data=data).to_json(index=False, orient='table')

    st.download_button(
        label='Download JSON',
        file_name='comparison-results.json',
        mime='application/json',
        data=json_file,
    )

将数据转换为CSV格式的过程有所不同，因为您需要使用koude25 method。

def save_to_csv(data: list):
    csv_file = pd.DataFrame(data=data).to_csv(index=False)

    st.download_button(
        label="Download CSV",
        file_name='comparison-results.csv',
        mime='text/csv',
        data=csv_file
    )