Bug 161433

(In reply to comment #0) > There are three solutions: > > A) Use DerivedSources.make (Bug 130339) > B) Generate Makefile for preprocess-idls.pl and generate-bindings.pl and > invoke 'make' > C) Implement Make-like tool in Python or Perl According to Ninja documentation, it seems to support dep file. https://ninja-build.org/manual.html#ref_rule We seem to have fourth choice: D) Generate bulid.ninja for preprocess-idls.pl and generate-bindings.pl and invoke 'ninja' or 'ninja-build'

I've finished a experimental prototyping. > https://github.com/fujii/webkit/pull/1 There are several commits of refrigerating CMake scripts. This is the important part: > https://github.com/fujii/webkit/commit/fffdb707fe123742a5440d5ec4a9c074f1c82d7c I've got well understanding thought this experiment. > C) Implement Make-like tool in Python or Perl This approach is better than others because more precise dependency updating checking is possible. For example, my patches solve the problem updating supplemental_dependency.tmp triggers regenerating all bindings. And, Perl is better than Python because generate-bindings.pl is written in Perl. If generate-bindings.pl will be converted to Perl Module, I can stop using fork&exec for speed and avoiding Windows quoting command problem. Ninja deletes all output files before executing a command. So, I need to use add_custom_target. Can I proceed?

(In reply to comment #2) > I've finished a experimental prototyping. > > > https://github.com/fujii/webkit/pull/1 > You have to upload the patch here, with the r? flag set so a reviewer can take a look into it. The patch should also contain changelogs (we have a tool to generate them). Check: https://webkit.org/contributing-code/ Very summarized the process for git users is the following: 1. Commit to a local git branch your patch. 2. Generate a changelog for it as follows: $ Tools/Scripts/prepare-ChangeLog -b $bugnumer -g $sha1hashofthecommit 3. Edit the changelogs and commit them amending the previous commit 4. Upload the patch here using this: $ Tools/Scripts/webkit-patch upload -g $sha1hashofthenewammendedcommit --suggest-reviewers --request-commit

(In reply to comment #3) > (In reply to comment #2) > > I've finished a experimental prototyping. > > > > > https://github.com/fujii/webkit/pull/1 > > > > You have to upload the patch here, with the r? flag set so a reviewer can > take a look into it. > > The patch should also contain changelogs (we have a tool to generate them). > > Check: https://webkit.org/contributing-code/ > > > Very summarized the process for git users is the following: > > 1. Commit to a local git branch your patch. > 2. Generate a changelog for it as follows: > $ Tools/Scripts/prepare-ChangeLog -b $bugnumer -g $sha1hashofthecommit > 3. Edit the changelogs and commit them amending the previous commit > 4. Upload the patch here using this: > $ Tools/Scripts/webkit-patch upload -g $sha1hashofthenewammendedcommit > --suggest-reviewers --request-commit If any of the previous commands fails, ensure that you have configured git-svn to track also the svn repository as documented here https://trac.webkit.org/wiki/UsingGitWithWebKit#Checkout and retry.

Thank you for the advice, Carlos. I'll do that.

Created attachment 290773 [details] WIP Patch

Created attachment 290803 [details] Patch

win EWS fails: > UIScriptController.idl > C:/cygwin/home/buildbot/WebKit/Tools/TestRunnerShared/UIScriptContext/Bindings/UIScriptController.idl > : No such file or directory at /home/buildbot/WebKit/Source/WebCore/bindings/scripts/generate-bindings.pl line 79. > Thread 1 terminated abnormally: Died at C:/cygwin/home/buildbot/WebKit/Source/WebCore/bindings/scripts/generate-bindings-all.pl line 157. > ...propagated at C:/cygwin/home/buildbot/WebKit/Source/WebCore/bindings/scripts/generate-bindings-all.pl line 162. > C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4.0\V140\Microsoft.CppCommon.targets(171,5): error MSB6006: "cmd.exe" exited with code 1. EWS uses Cygwin Perl. This problem seems a filename with trailing CR. > -- Found Perl: C:/cygwin/bin/perl.exe (found suitable version "5.22.2", minimum required is "5.10.0")

Created attachment 290810 [details] Patch

Created attachment 290919 [details] Patch Rebased the patch for r206904.

Comment on attachment 290919 [details] Patch View in context: https://bugs.webkit.org/attachment.cgi?id=290919&action=review I think this looks good, but I'd like Carlos to give final approval. > Source/WebCore/ChangeLog:24 > + * bindings/scripts/generate-bindings-all.pl: Added. Is the original "generate-bindings.pl" still need after this change?

Thank you for reviewing my patch. (In reply to comment #11) > Is the original "generate-bindings.pl" still need after this change? generate-bindings.pl is still used in non-CMake port (Mac port) and run-bindings-tests. And, I can not combine these two scripts without tricks because generate-bindings.pl uses global variables which prevents multi-threading. I think it's good idea to do refactoring. If generate-bindings.pl will be converted to Perl Module, I can stop using fork&exec for speed and avoiding Windows quoting command problem.

Comment on attachment 290919 [details] Patch Since Carlos hasn't objected, I think we should move forward. We can always fix up afterwards if there are any issues.

Comment on attachment 290919 [details] Patch Clearing flags on attachment: 290919 Committed r206972: <http://trac.webkit.org/changeset/206972>

All reviewed patches have been landed. Closing bug.

Hm, it's very noisy: [96/3651] Generate bindings (WebCoreBindings) Preprocess IDL WebKitPlaybackTargetAvailabilityEvent.idl BatteryManager.idl NavigatorBattery.idl DOMWindowFetch.idl FetchBody.idl FetchHeaders.idl FetchRequest.idl FetchResponse.idl WorkerGlobalScopeFetch.idl Coordinates.idl Geolocation.idl Geoposition.idl NavigatorGeolocation.idl PositionCallback.idl PositionError.idl PositionErrorCallback.idl DOMWindowIndexedDatabase.idl IDBCursor.idl IDBCursorWithValue.idl IDBDatabase.idl IDBFactory.idl IDBIndex.idl IDBKeyRange.idl IDBObjectStore.idl IDBOpenDBRequest.idl IDBRequest.idl IDBTransaction.idl IDBVersionChangeEvent.idl WorkerGlobalScopeIndexedDatabase.idl UIRequestEvent.idl MediaControlsHost.idl HTMLMediaElementMediaSession.idl MediaRemoteControls.idl MediaSession.idl DOMURLMediaSource.idl MediaSource.idl SourceBuffer.idl SourceBufferList.idl VideoPlaybackQuality.idl DOMURLMediaStream.idl HTMLMediaElementMediaStream.idl MediaDeviceInfo.idl MediaDevices.idl MediaStream.idl MediaStreamEvent.idl MediaStreamTrack.idl MediaStreamTrackEvent.idl MediaTrackConstraint.idl MediaTrackConstraintSet.idl MediaTrackConstraints.idl MediaTrackSupportedConstraints.idl NavigatorMediaDevices.idl NavigatorUserMedia.idl OverconstrainedError.idl OverconstrainedErrorEvent.idl RTCConfiguration.idl RTCDTMFSender.idl RTCDTMFToneChangeEvent.idl RTCDataChannel.idl RTCDataChannelEvent.idl RTCIceCandidate.idl RTCIceCandidateEvent.idl RTCIceServer.idl RTCPeerConnection.idl RTCRtpReceiver.idl RTCRtpSender.idl RTCRtpTransceiver.idl RTCSessionDescription.idl RTCStatsReport.idl RTCStatsResponse.idl RTCTrackEvent.idl NavigatorContentUtils.idl DOMWindowNotifications.idl Notification.idl NotificationCenter.idl NotificationPermissionCallback.idl WorkerGlobalScopeNotifications.idl DeviceProximityEvent.idl DOMWindowSpeechSynthesis.idl SpeechSynthesis.idl SpeechSynthesisEvent.idl SpeechSynthesisUtterance.idl SpeechSynthesisVoice.idl ByteLengthQueuingStrategy.idl CountQueuingStrategy.idl ReadableStream.idl ReadableStreamDefaultController.idl ReadableStreamDefaultReader.idl ReadableStreamSource.idl WritableStream.idl NavigatorVibration.idl AnalyserNode.idl AudioBuffer.idl AudioBufferCallback.idl AudioBufferSourceNode.idl AudioContext.idl AudioDestinationNode.idl AudioListener.idl AudioNode.idl AudioParam.idl AudioProcessingEvent.idl BiquadFilterNode.idl ChannelMergerNode.idl ChannelSplitterNode.idl ConvolverNode.idl DelayNode.idl DynamicsCompressorNode.idl GainNode.idl MediaElementAudioSourceNode.idl MediaStreamAudioDestinationNode.idl MediaStreamAudioSourceNode.idl OfflineAudioCompletionEvent.idl OfflineAudioContext.idl OscillatorNode.idl PannerNode.idl PeriodicWave.idl ScriptProcessorNode.idl WaveShaperNode.idl DOMWindowWebDatabase.idl Database.idl DatabaseCallback.idl SQLError.idl SQLException.idl SQLResultSet.idl SQLResultSetRowList.idl SQLStatementCallback.idl SQLStatementErrorCallback.idl SQLTransaction.idl SQLTransactionCallback.idl SQLTransactionErrorCallback.idl CloseEvent.idl WebSocket.idl Animatable.idl AnimationEffect.idl AnimationTimeline.idl DocumentAnimation.idl DocumentTimeline.idl KeyframeEffect.idl WebAnimation.idl CryptoKey.idl CryptoKeyPair.idl SubtleCrypto.idl WebKitSubtleCrypto.idl CSSCharsetRule.idl CSSFontFaceLoadEvent.idl CSSFontFaceRule.idl CSSImportRule.idl CSSKeyframeRule.idl CSSKeyframesRule.idl CSSMediaRule.idl CSSPageRule.idl CSSPrimitiveValue.idl CSSRule.idl CSSRuleList.idl CSSStyleDeclaration.idl CSSStyleRule.idl CSSStyleSheet.idl CSSSupportsRule.idl CSSUnknownRule.idl CSSValue.idl CSSValueList.idl Counter.idl DOMCSSNamespace.idl FontFace.idl FontFaceSet.idl MediaList.idl MediaQueryList.idl MediaQueryListListener.idl RGBColor.idl Rect.idl StyleMedia.idl StyleSheet.idl StyleSheetList.idl WebKitCSSFilterValue.idl WebKitCSSMatrix.idl WebKitCSSRegionRule.idl WebKitCSSTransformValue.idl WebKitCSSViewportRule.idl AnimationEvent.idl Attr.idl BeforeLoadEvent.idl BeforeUnloadEvent.idl CDATASection.idl CharacterData.idl ChildNode.idl ClientRect.idl ClientRectList.idl ClipboardEvent.idl Comment.idl CompositionEvent.idl CustomElementRegistry.idl CustomEvent.idl DOMCoreException.idl DOMError.idl DOMImplementation.idl DOMNamedFlowCollection.idl DOMStringList.idl DOMStringMap.idl DataTransfer.idl DataTransferItem.idl DataTransferItemList.idl DeviceMotionEvent.idl DeviceOrientationEvent.idl Document.idl DocumentFragment.idl DocumentOrShadowRoot.idl DocumentType.idl Element.idl ErrorEvent.idl Event.idl EventInit.idl EventModifierInit.idl EventTarget.idl FocusEvent.idl GlobalEventHandlers.idl HashChangeEvent.idl InputEvent.idl KeyboardEvent.idl MessageChannel.idl MessageEvent.idl MessagePort.idl MouseEvent.idl MutationEvent.idl MutationObserver.idl MutationRecord.idl NamedNodeMap.idl Node.idl NodeFilter.idl NodeIterator.idl NodeList.idl NonDocumentTypeChildNode.idl NonElementParentNode.idl OverflowEvent.idl PageTransitionEvent.idl ParentNode.idl PopStateEvent.idl ProcessingInstruction.idl ProgressEvent.idl Range.idl RequestAnimationFrameCallback.idl SecurityPolicyViolationEvent.idl ShadowRoot.idl Slotable.idl StringCallback.idl Text.idl TextEvent.idl Touch.idl TouchEvent.idl TouchList.idl TransitionEvent.idl TreeWalker.idl UIEvent.idl UIEventInit.idl WebKitAnimationEvent.idl WebKitNamedFlow.idl WebKitTransitionEvent.idl WheelEvent.idl XMLDocument.idl Blob.idl File.idl FileError.idl FileException.idl FileList.idl FileReader.idl FileReaderSync.idl DOMFormData.idl DOMTokenList.idl DOMURL.idl HTMLAllCollection.idl HTMLAnchorElement.idl HTMLAppletElement.idl HTMLAreaElement.idl HTMLAttachmentElement.idl HTMLAudioElement.idl HTMLBRElement.idl HTMLBaseElement.idl HTMLBodyElement.idl HTMLButtonElement.idl HTMLCanvasElement.idl HTMLCollection.idl HTMLDListElement.idl HTMLDataElement.idl HTMLDataListElement.idl HTMLDetailsElement.idl HTMLDirectoryElement.idl HTMLDivElement.idl HTMLDocument.idl HTMLElement.idl HTMLEmbedElement.idl HTMLFieldSetElement.idl HTMLFontElement.idl HTMLFormControlsCollection.idl HTMLFormElement.idl HTMLFrameElement.idl HTMLFrameSetElement.idl HTMLHRElement.idl HTMLHeadElement.idl HTMLHeadingElement.idl HTMLHtmlElement.idl HTMLHyperlinkElementUtils.idl HTMLIFrameElement.idl HTMLImageElement.idl HTMLInputElement.idl HTMLKeygenElement.idl HTMLLIElement.idl HTMLLabelElement.idl HTMLLegendElement.idl HTMLLinkElement.idl HTMLMapElement.idl HTMLMarqueeElement.idl HTMLMediaElement.idl HTMLMenuElement.idl HTMLMetaElement.idl HTMLMeterElement.idl HTMLModElement.idl HTMLOListElement.idl HTMLObjectElement.idl HTMLOptGroupElement.idl HTMLOptionElement.idl HTMLOptionsCollection.idl HTMLOutputElement.idl HTMLParagraphElement.idl HTMLParamElement.idl HTMLPictureElement.idl HTMLPreElement.idl HTMLProgressElement.idl HTMLQuoteElement.idl HTMLScriptElement.idl HTMLSelectElement.idl HTMLSlotElement.idl HTMLSourceElement.idl HTMLSpanElement.idl HTMLStyleElement.idl HTMLTableCaptionElement.idl HTMLTableCellElement.idl HTMLTableColElement.idl HTMLTableElement.idl HTMLTableRowElement.idl HTMLTableSectionElement.idl HTMLTemplateElement.idl HTMLTextAreaElement.idl HTMLTimeElement.idl HTMLTitleElement.idl HTMLTrackElement.idl HTMLUListElement.idl HTMLUnknownElement.idl HTMLVideoElement.idl ImageData.idl MediaController.idl MediaError.idl RadioNodeList.idl TextMetrics.idl TimeRanges.idl URLSearchParams.idl URLUtils.idl ValidityState.idl VoidCallback.idl CanvasGradient.idl CanvasPath.idl CanvasPattern.idl CanvasProxy.idl CanvasRenderingContext2D.idl DOMPath.idl CommandLineAPIHost.idl InspectorFrontendHost.idl DOMApplicationCache.idl BarProp.idl Crypto.idl DOMSelection.idl DOMWindow.idl EventSource.idl GlobalCrypto.idl History.idl Location.idl Navigator.idl NavigatorConcurrentHardware.idl NavigatorID.idl NavigatorLanguage.idl NavigatorOnLine.idl Performance.idl PerformanceEntry.idl PerformanceMark.idl PerformanceMeasure.idl PerformanceNavigation.idl PerformanceResourceTiming.idl PerformanceTiming.idl Screen.idl ScrollToOptions.idl WebKitPoint.idl WindowEventHandlers.idl WindowOrWorkerGlobalScope.idl WorkerNavigator.idl DOMMimeTypeArray.idl DOMMimeType.idl DOMPlugin.idl DOMPluginArray.idl Storage.idl StorageEvent.idl AbstractWorker.idl DedicatedWorkerGlobalScope.idl Worker.idl WorkerGlobalScope.idl WorkerLocation.idl DOMParser.idl XMLHttpRequest.idl XMLHttpRequestEventTarget.idl XMLHttpRequestProgressEvent.idl XMLHttpRequestUpload.idl XMLSerializer.idl XPathEvaluator.idl XPathException.idl XPathExpression.idl XPathNSResolver.idl XPathResult.idl XSLTProcessor.idl SVGAElement.idl SVGAltGlyphDefElement.idl SVGAltGlyphElement.idl SVGAltGlyphItemElement.idl SVGAngle.idl SVGAnimateColorElement.idl SVGAnimateElement.idl SVGAnimateMotionElement.idl SVGAnimateTransformElement.idl SVGAnimatedAngle.idl SVGAnimatedBoolean.idl SVGAnimatedEnumeration.idl SVGAnimatedInteger.idl SVGAnimatedLength.idl SVGAnimatedLengthList.idl SVGAnimatedNumber.idl SVGAnimatedNumberList.idl SVGAnimatedPreserveAspectRatio.idl SVGAnimatedRect.idl SVGAnimatedString.idl SVGAnimatedTransformList.idl SVGAnimationElement.idl SVGCircleElement.idl SVGClipPathElement.idl SVGColor.idl SVGComponentTransferFunctionElement.idl SVGCursorElement.idl SVGDefsElement.idl SVGDescElement.idl SVGDocument.idl SVGElement.idl SVGEllipseElement.idl SVGException.idl SVGExternalResourcesRequired.idl SVGFEBlendElement.idl SVGFEColorMatrixElement.idl SVGFEComponentTransferElement.idl SVGFECompositeElement.idl SVGFEConvolveMatrixElement.idl SVGFEDiffuseLightingElement.idl SVGFEDisplacementMapElement.idl SVGFEDistantLightElement.idl SVGFEDropShadowElement.idl SVGFEFloodElement.idl SVGFEFuncAElement.idl SVGFEFuncBElement.idl SVGFEFuncGElement.idl SVGFEFuncRElement.idl SVGFEGaussianBlurElement.idl SVGFEImageElement.idl SVGFEMergeElement.idl SVGFEMergeNodeElement.idl SVGFEMorphologyElement.idl SVGFEOffsetElement.idl SVGFEPointLightElement.idl SVGFESpecularLightingElement.idl SVGFESpotLightElement.idl SVGFETileElement.idl SVGFETurbulenceElement.idl SVGFilterElement.idl SVGFilterPrimitiveStandardAttributes.idl SVGFitToViewBox.idl SVGFontElement.idl SVGFontFaceElement.idl SVGFontFaceFormatElement.idl SVGFontFaceNameElement.idl SVGFontFaceSrcElement.idl SVGFontFaceUriElement.idl SVGForeignObjectElement.idl SVGGElement.idl SVGGlyphElement.idl SVGGlyphRefElement.idl SVGGradientElement.idl SVGGraphicsElement.idl SVGHKernElement.idl SVGImageElement.idl SVGLength.idl SVGLengthList.idl SVGLineElement.idl SVGLinearGradientElement.idl SVGMPathElement.idl SVGMarkerElement.idl SVGMaskElement.idl SVGMatrix.idl SVGMetadataElement.idl SVGMissingGlyphElement.idl SVGNumber.idl SVGNumberList.idl SVGPaint.idl SVGPathElement.idl SVGPathSeg.idl SVGPathSegArcAbs.idl SVGPathSegArcRel.idl SVGPathSegClosePath.idl SVGPathSegCurvetoCubicAbs.idl SVGPathSegCurvetoCubicRel.idl SVGPathSegCurvetoCubicSmoothAbs.idl SVGPathSegCurvetoCubicSmoothRel.idl SVGPathSegCurvetoQuadraticAbs.idl SVGPathSegCurvetoQuadraticRel.idl SVGPathSegCurvetoQuadraticSmoothAbs.idl SVGPathSegCurvetoQuadraticSmoothRel.idl SVGPathSegLinetoAbs.idl SVGPathSegLinetoHorizontalAbs.idl SVGPathSegLinetoHorizontalRel.idl SVGPathSegLinetoRel.idl SVGPathSegLinetoVerticalAbs.idl SVGPathSegLinetoVerticalRel.idl SVGPathSegList.idl SVGPathSegMovetoAbs.idl SVGPathSegMovetoRel.idl SVGPatternElement.idl SVGPoint.idl SVGPointList.idl SVGPolygonElement.idl SVGPolylineElement.idl SVGPreserveAspectRatio.idl SVGRadialGradientElement.idl SVGRect.idl SVGRectElement.idl SVGRenderingIntent.idl SVGSVGElement.idl SVGScriptElement.idl SVGSetElement.idl SVGStopElement.idl SVGStringList.idl SVGStyleElement.idl SVGSwitchElement.idl SVGSymbolElement.idl SVGTRefElement.idl SVGTSpanElement.idl SVGTests.idl SVGTextContentElement.idl SVGTextElement.idl SVGTextPathElement.idl SVGTextPositioningElement.idl SVGTitleElement.idl SVGTransform.idl SVGTransformList.idl SVGURIReference.idl SVGUnitTypes.idl SVGUseElement.idl SVGVKernElement.idl SVGViewElement.idl SVGViewSpec.idl SVGZoomAndPan.idl SVGZoomEvent.idl ANGLEInstancedArrays.idl EXTBlendMinMax.idl EXTFragDepth.idl EXTShaderTextureLOD.idl EXTTextureFilterAnisotropic.idl EXTsRGB.idl OESElementIndexUint.idl OESStandardDerivatives.idl OESTextureFloat.idl OESTextureFloatLinear.idl OESTextureHalfFloat.idl OESTextureHalfFloatLinear.idl OESVertexArrayObject.idl WebGL2RenderingContext.idl WebGLActiveInfo.idl WebGLBuffer.idl WebGLCompressedTextureATC.idl WebGLCompressedTexturePVRTC.idl WebGLCompressedTextureS3TC.idl WebGLContextAttributes.idl WebGLContextEvent.idl WebGLDebugRendererInfo.idl WebGLDebugShaders.idl WebGLDepthTexture.idl WebGLDrawBuffers.idl WebGLFramebuffer.idl WebGLLoseContext.idl WebGLProgram.idl WebGLQuery.idl WebGLRenderbuffer.idl WebGLRenderingContext.idl WebGLRenderingContextBase.idl WebGLSampler.idl WebGLShader.idl WebGLShaderPrecisionFormat.idl WebGLSync.idl WebGLTexture.idl WebGLTransformFeedback.idl WebGLUniformLocation.idl WebGLVertexArrayObject.idl WebGLVertexArrayObjectOES.idl AudioTrack.idl AudioTrackList.idl DataCue.idl TextTrack.idl TextTrackCue.idl TextTrackCueList.idl TextTrackList.idl TrackEvent.idl VTTCue.idl VTTRegion.idl VTTRegionList.idl VideoTrack.idl VideoTrackList.idl UserMessageHandler.idl UserMessageHandlersNamespace.idl WebKitNamespace.idl [110/3651] Generate bindings (WebKitTestRunnerInjectedBundleBindings) AccessibilityController.idl AccessibilityTextMarker.idl AccessibilityTextMarkerRange.idl AccessibilityUIElement.idl EventSendingController.idl GCController.idl TestRunner.idl TextInputController.idl [779/3651] Generate bindings (WebKitTestRunnerBindings) UIScriptController.idl [2995/3651] Generate bindings (WebCoreTestSupportBindings) GCObservation.idl InternalSettings.idl Internals.idl MallocStatistics.idl MemoryInfo.idl MockContentFilterSettings.idl MockPageOverlay.idl TypeConversions.idl InternalSettingsGenerated.idl We previously had a nice clean build. Can we make it not print, please?

It also seems to be doing some commands now always, even on incremental builds when nothing changed [0/935] Generate bindings (WebCoreBindings) [2/38] Generate bindings (WebKitTestRunnerBindings) [3/36] Generate bindings (WebCoreTestSupportBindings) [4/24] Generate bindings (WebKitTestRunnerInjectedBundleBindings)

We are seeing some EWS flakiness on Windows as well. Could you please look into this ASAP? Here's the EWS error: Thread 2 terminated abnormally: Died at C:/cygwin/home/buildbot/WebKit/Source/WebCore/bindings/scripts/generate-bindings-all.pl line 157. Here's that line in the perl: system($perl, "$scriptDir/generate-bindings.pl", @args, $file) == 0 or die; I'm not sure why this is failing. I got this to happen in a local Windows build, but when I built a second time it seemed to bypass the error.

Re-opened since this is blocked by bug 163227

I'm (regretfully) rolling this change out, because it has made EWS flaky and is causing some unwanted noise for GTK builds. I don't have enough deep CMake skills to understand why this is happening, but I'd like to back it out until Fujii has chance to review the issue.

(In reply to comment #20) > I'm (regretfully) rolling this change out, because it has made EWS flaky and > is causing some unwanted noise for GTK builds. I'm sorry for causing the serious problem and thank you for rolling back, Brent. (In reply to comment #16) > Hm, it's very noisy: > > [96/3651] Generate bindings (WebCoreBindings) > Preprocess IDL (..) > We previously had a nice clean build. Can we make it not print, please? I think it should output something because it takes long time to generate all bindings. I'll improve the progress output as Ninja does. (In reply to comment #17) > It also seems to be doing some commands now always, even on incremental > builds when nothing changed > > [0/935] Generate bindings (WebCoreBindings) > [2/38] Generate bindings (WebKitTestRunnerBindings) > [3/36] Generate bindings (WebCoreTestSupportBindings) > [4/24] Generate bindings (WebKitTestRunnerInjectedBundleBindings) This is intended. It checks which IDL needs to be regenerated by checking timestamp as well as generate-forwarding-headers.pl does.

I've succeeded to reproduce this flaky problem. The reproduction ratio of this problem depends on which Perl is used and which version of Cygwin is used. 64bit ActivePerl : never happen 64bit latest Cygwin Perl: never happen 32bit latest Cygwin Perl: I saw the problem only once 32bit 5-month-old Cygwin Perl: about 10% 32bit 3-year-old Cygwin Perl: never happen I dumped $! and $?. > Can't exec "/usr/bin/perl": Bad address at C:/home/0000134108/work/webkit/webkit1/Source/WebCore/bindings/scripts/generate-bindings-all.pl line 159. > Thread 8 terminated abnormally: generate-bindings.pl fails: ($!=Bad address)($?=-1) at C:/home/0000134108/work/webkit/webkit1/Source/WebCore/bindings/scripts/generate-bindings-all.pl line 159. > ...propagated at C:/home/0000134108/work/webkit/webkit1/Source/WebCore/bindings/scripts/generate-bindings-all.pl line 164. >C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4.0\V140\Microsoft.CppCommon.targets(171,5): error MSB6006: "cmd.exe" exited with code 1. Sometimes different error message and no error message: > SVGScriptElement.idl > SVGStringList.idl > 1 [main] perl 3588 cygwin_exception::open_stackdumpfile: Dumping stack trace to perl.exe.stackdump >C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4.0\V140\Microsoft.CppCommon.targets(171,5): error MSB6006: "cmd.exe" exited with code 1536. This seems Cygwin fork failure problem. https://cygwin.com/faq/faq.html#faq.using.fixing-fork-failures I'd like to request installing ActiveState Perl on BuildBot as described in this page <https://webkit.org/webkit-on-windows/>. And, Uninstall Cygwin because cmake script append a path to cygwin. <https://trac.webkit.org/browser/trunk/Source/cmake/WebKitCommon.cmake?rev=207060#L13>

Hi Alex, Blent: You added Cygwin path to CMAKE_PROGRAM_PATH in Bug 152937 for Apple internal Windows build. Is there a reason to need to use Cygwin tools yet? Why don't you use Windows native tools? I have a trouble with Cygwin Perl.

CygwinDownloader has perl-libwin32. <https://trac.webkit.org/browser/trunk/Tools/CygwinDownloader/cygwin-downloader.py?rev=167430#L63> It seems to be able to use Win32::Job perl module for the workaround. Win32::Job http://search.cpan.org/~gsar/libwin32-0.191/Job/Job.pm I'll try.

Created attachment 291471 [details] Patch * Use Win32::Job for the workaround * Improve the progress output such like Ninja does

Comment on attachment 291471 [details] Patch Win EWS failed: > Can't locate Win32/Job.pm in @INC (you may need to install the Win32::Job module) (@INC contains: /home/buildbot/WebKit/Source/WebCore/bindings/scripts /usr/lib/perl5/site_perl/5.22/i686-cygwin-threads-64int /usr/lib/perl5/site_perl/5.22 /usr/lib/perl5/vendor_perl/5.22/i686-cygwin-threads-64int /usr/lib/perl5/vendor_perl/5.22 /usr/lib/perl5/5.22/i686-cygwin-threads-64int /usr/lib/perl5/5.22 .) at /usr/lib/perl5/5.22/if.pm line 13. No luck.

(In reply to comment #23) > Hi Alex, Blent: > > You added Cygwin path to CMAKE_PROGRAM_PATH in Bug 152937 for > Apple internal Windows build. Is there a reason to need to use > Cygwin tools yet? Why don't you use Windows native tools? I have > a trouble with Cygwin Perl. Our internal build needs to continue to use Cygwin. The WinCairo bot doesn't even have Cygwin installed. I agree that's the preferred way to do development on Windows, but we need to continue to support Cygwin builds.

Created attachment 291573 [details] Minimal test case for Cygwin Perl This is a minimal test case for Cygwin Perl. Cygwin Perl fails to execute a external command with 'system' function in multi-threading. This problem seems that 'system' of Cygwin Perl is not thread-safe. Not a Cygwin fork problem. It works fine if I use fork&exec instread of 'system' even in Cygwin Perl. I'll rewrite the patch.

Created attachment 291589 [details] Patch * Use fork&exec if Cygwin Perl is used * Fix a bug of getting TTY width

Created attachment 291790 [details] Patch * Merged r207311

Comment on attachment 291790 [details] Patch View in context: https://bugs.webkit.org/attachment.cgi?id=291790&action=review Looks good. I'm running it on a local Windows build to make sure it works on our standard setup. > Source/WebCore/bindings/scripts/generate-bindings-all.pl:83 > + executeCommand($perl, "$scriptDir/preprocess-idls.pl", @args) == 0 or die; This is fine, but I prefer "File::Spec->catfile" and related so we don't have to worry about path separator differences for WIndows versus Unix. > Source/WebCore/bindings/scripts/generate-bindings-all.pl:146 > + # print "$_ is newer than $oldestObject\n"; Should remove this debugging statement.

This worked very nicely on my local Cygwin-based build! Fujii, could you please correct those two minor comments I made when you land this?

Created attachment 292147 [details] Patch Thank you for r+ and testing my patch. I updated the patch. Please give me cq+. * Use File::Spec->catfile for preprocess-idls.pl and generate-bindings.pl * Removed the debugging statement * Fix the problem 'stty is not recognized' is output in Windows Ninja Win32 Perl build.

Comment on attachment 292147 [details] Patch Clearing flags on attachment: 292147 Committed r207617: <http://trac.webkit.org/changeset/207617>

All reviewed patches have been landed. Closing bug.

Coming CMake 3.7 will finally have DEPFILE argument of add_custom_command only for Ninja generator. https://cmake.org/cmake/help/v3.7/command/add_custom_command.html