如何使用不同的作业参数多次运行弹簧批处理作业?(How to run spring batch job multiple times with different job parameters?)
我们用单步创建了一个简单的弹簧批处理作业。 有自定义实现的ItemReader和ItemWriter。 ItemReader从job参数获取初始数据。 当作为独立的java进程运行时,批处理运行完美。 但我们想要的是在一些服务器上托管批处理。 因此,我们已经创建了REST服务来初始化批处理。 该服务调用作业URL并传递一些参数。 此参数作为作业参数传递给批处理。 调用一个参数时,服务和作业运行正常。
但是当我们不止一次调用该服务(两次用于测试目的)时,批处理行为奇怪。 我们正在传递不同的工作参数。 但是当第二个作业初始化执行开始时,ItemReader接收的作业参数值与第一次执行的作业参数值相同。 并且两个执行都相互干扰,共享数据库连接,干扰检索的数据等。
我们已经尝试将restartable参数设置为false但它不起作用。 我们还尝试了以下解决方案:
我们可以创建同一个java(spring)批处理作业的多个实例吗?
上面的解决方案开始在JBoss中给出“中断尝试锁定”错误。
在进一步调查中,我们发现ItemReader只被初始化一次。 这就是为什么它获得相同的作业参数值并干扰先前的执行。
编辑
以下是作业配置:<bean id="jobLauncher" class="org.springframework.batch.core.launch.support.SimpleJobLauncher"> <property name="jobRepository" ref="jobRepository" />
<job id="jobid" restartable="false"> <step id="step1"> <tasklet> <chunk reader="reader" writer="writer" commit-interval="2"> </chunk> </tasklet> </step> </job>
以下是启动作业的代码段:
JobLauncher jobLauncher = (JobLauncher) context.getBean("jobLauncher"); Job job = (Job) context.getBean("jobid"); try { JobParameters param = new JobParametersBuilder().addString("key","value").toJobParameters(); JobExecution execution = jobLauncher.run(job, param); } catch (Exception e) { e.printStackTrace(); }
有人可以建议一些解决方案吗? 我错过了一步的配置吗? 提前致谢。
We have created a simple spring batch job with single step. There are custom implemented ItemReader and ItemWriter. The ItemReader gets the initial data from job parameter. The batch runs perfectly when run as a standalone java process. But what we want is to host the batch on some server. Therefore, we have created REST service to initialize the batch. The service calls the job URL and passes some parameter. This parameter is passed as job parameter to the batch. The service and job run fine when it is called for one parameter.
But when we call the service more than once (twice for testing purpose), the batch behaves strangely. We are passing different job parameters. But when the execution starts for second job initialization, the job parameter value which is received by the ItemReader is the same as the one for the first execution. And both execution interfere with each other, sharing database connection, interfering with data retrieved etc.
We have tried setting the restartable parameter to false but it didn't work. We have also tried the following solution:
Can we create multiple instances of a same java(spring) batch job?
The above solution started giving "Interrupted attempting lock" error in JBoss.
On further investigation we found that ItemReader is getting initialized only once. That is why it is getting same job parameter value and is interfering with the previous execution.
EDIT
Following is the job configuration:<bean id="jobLauncher" class="org.springframework.batch.core.launch.support.SimpleJobLauncher"> <property name="jobRepository" ref="jobRepository" />
<job id="jobid" restartable="false"> <step id="step1"> <tasklet> <chunk reader="reader" writer="writer" commit-interval="2"> </chunk> </tasklet> </step> </job>
Following is the code snippet to launch the job:
JobLauncher jobLauncher = (JobLauncher) context.getBean("jobLauncher"); Job job = (Job) context.getBean("jobid"); try { JobParameters param = new JobParametersBuilder().addString("key","value").toJobParameters(); JobExecution execution = jobLauncher.run(job, param); } catch (Exception e) { e.printStackTrace(); }
Can anyone please suggest some solution? Am I missing some configuration for the step? Thanks in advance.
原文:https://stackoverflow.com/questions/31313826
最满意答案
你非常接近; 你只需要在最后一行使用列表理解而不是生成器表达式。
Date_List = (datetime.datetime.strptime(i, "%Y-%m-%d %H:%M:%S") for i in Date_List) Date_List = [datetime.datetime.strftime(i, "%m-%d-%Y") for i in Date_List]
我会像这样清理它:
from datetime import datetime from pprint import pprint timestamps = [ '2015-08-24 00:00:00', '2015-08-30 00:00:00', '2015-08-22 00:00:00', '2015-08-21 00:00:00', '2015-08-25 00:00:00', '2015-08-29 00:00:00', ] dates = (datetime.strptime(ts, '%Y-%m-%d %H:%M:%S') for ts in timestamps) date_strings = [datetime.strftime(d, '%m-%d-%Y') for d in dates] pprint(date_strings)
输出:
['08-24-2015', '08-30-2015', '08-22-2015', '08-21-2015', '08-25-2015', '08-29-2015']
这是一个稍微更一般化的方法:
from datetime import datetime from pprint import pprint def convert_timestamp(ts, from_pattern, to_pattern): dt = datetime.strptime(ts, from_pattern) return datetime.strftime(dt, to_pattern) timestamps = [ '2015-08-24 00:00:00', '2015-08-30 00:00:00', '2015-08-22 00:00:00', '2015-08-21 00:00:00', '2015-08-25 00:00:00', '2015-08-29 00:00:00', ] date_strings = [convert_timestamp(ts, '%Y-%m-%d %H:%M:%S', '%m-%d-%Y') for ts in timestamps] pprint(date_strings)
输出:
['08-24-2015', '08-30-2015', '08-22-2015', '08-21-2015', '08-25-2015', '08-29-2015']
You're very close; you just need to use a list comprehension on the last line instead of a generator expression.
Date_List = (datetime.datetime.strptime(i, "%Y-%m-%d %H:%M:%S") for i in Date_List) Date_List = [datetime.datetime.strftime(i, "%m-%d-%Y") for i in Date_List]
I would clean it up like so:
from datetime import datetime from pprint import pprint timestamps = [ '2015-08-24 00:00:00', '2015-08-30 00:00:00', '2015-08-22 00:00:00', '2015-08-21 00:00:00', '2015-08-25 00:00:00', '2015-08-29 00:00:00', ] dates = (datetime.strptime(ts, '%Y-%m-%d %H:%M:%S') for ts in timestamps) date_strings = [datetime.strftime(d, '%m-%d-%Y') for d in dates] pprint(date_strings)
Output:
['08-24-2015', '08-30-2015', '08-22-2015', '08-21-2015', '08-25-2015', '08-29-2015']
Here's a slightly more generalized way to do it:
from datetime import datetime from pprint import pprint def convert_timestamp(ts, from_pattern, to_pattern): dt = datetime.strptime(ts, from_pattern) return datetime.strftime(dt, to_pattern) timestamps = [ '2015-08-24 00:00:00', '2015-08-30 00:00:00', '2015-08-22 00:00:00', '2015-08-21 00:00:00', '2015-08-25 00:00:00', '2015-08-29 00:00:00', ] date_strings = [convert_timestamp(ts, '%Y-%m-%d %H:%M:%S', '%m-%d-%Y') for ts in timestamps] pprint(date_strings)
Output:
['08-24-2015', '08-30-2015', '08-22-2015', '08-21-2015', '08-25-2015', '08-29-2015']
相关问答
更多-
您可以使用字典将年份和月份保存为键,将列表中的相对天数保存为值,然后您可以对按年份和月份分类的项目执行任何操作。 >>> dates = [['2014', '11', '14'], ['2014', '10', '13'], ['2014', '10', '01'], ['2014', '12', '01'], ['2013', '12', '01'], ['2013', '12', '09'], ['2013', '10', '01'], ['2013', '10', '05'], ['2013', ' ...
-
如何获得python中两个日期之间的所有日期(月,日和年)?(How to get all dates (month, day and year) between two dates in python?)[2022-03-09]
你不必重新发明轮子。 只需将字符串解析为datetime对象并让python为您做数学运算: from dateutil import rrule from datetime import datetime a = '20120525' b = '20120627' for dt in rrule.rrule(rrule.DAILY, dtstart=datetime.strptime(a, '%Y%m%d'), u ... -
有一个非常简单的解决方案: day_of_year = datetime.now().timetuple().tm_yday There is a very simple solution: from datetime import datetime day_of_year = datetime.now().timetuple().tm_yday
-
如果常规方法不起作用,您可以始终回避编写自己的解析器。 创建一个接受parse_dates列的parse_dates并返回datetime并使用date_parser添加该函数。 所以类似于: df = pd.read_csv(file, header=None, index_col='datetime', parse_dates={'datetime': [1,2,3,4]}, date_parser=lambda x: pd.dat ...
-
更改您的parser并像这样使用它: import pandas as pd from datetime import datetime def parse(year, month, day, hour, minute, second): return year+ '-' +month+ '-' +day+ ' ' +hour+ ':' +minute+ ':' +second df = pd.read_csv('test.csv', parse_dates={'Date/Time':['yea ...
-
日期字符串 - 12/31/2012 它匹配 - MM/dd/yyyy d - 每月的某一天 M - 一年中的月份 年 - 年 ... SimpleDateFormat sdf = new SimpleDateFormat("MM/dd/yyyy"); Date String - 12/31/2012 It matches with - MM/dd/yyyy d - Day of the month M - Month in year y - Year ... SimpleDateFormat sdf = ...
-
Python - 按月分组日期(Python - Group Dates by Month)[2022-07-07]
我建议使用pandas : import pandas as pd dates = ['28/02/2018', '01/03/2018', '16/03/2018', '17/05/2018'] s = pd.to_datetime(pd.Series(dates), format='%d/%m/%Y') s.index = s.dt.to_period('m') s = s.groupby(level=0).size() s = s.reindex(pd.period_range(s.index ... -
以下是Frank在评论中提到的使用lubridate提取器和替换功能的解决方案。 关键的是mday() yday<- , mday()和month() ,它们分别设置日期的年份,获取日期的月份,并获取日期的月份。 8秒的运行时间对我来说似乎是可以接受的,尽管我确信一些优化可以减少这种情况,尽管可能会失去一般性。 另请注意使用case_when确保闰年2月29日之后正确的天数。 编辑:这是一个明显更快的解决方案。 您可以将DOY映射到一年的月和双周,然后将left_join到主表。 运行时间为0.36秒,因为 ...
-
只需将时间与-或T分开,前三个元素应对应于年,月和日列,将其与其他两列连接将得到您需要的内容: pd.concat([df.drop('time', axis = 1), (df.time.str.split("-|T").str[:3].apply(pd.Series) .rename(columns={0:'year', 1:'month', 2:'day'}))], axis = 1) 接近@ nlassaux方法的另一种方法是: df['time'] = ...
-
将字符串中的日期列表转换为Python中的Month,Day和Year(Converting a list of dates in string to Month, Day, Year in Python)[2023-03-10]
你非常接近; 你只需要在最后一行使用列表理解而不是生成器表达式。 Date_List = (datetime.datetime.strptime(i, "%Y-%m-%d %H:%M:%S") for i in Date_List) Date_List = [datetime.datetime.strftime(i, "%m-%d-%Y") for i in Date_List] 我会像这样清理它: from datetime import datetime from pprint import ppri ...